,
10 tweets,
2 min read
Read on Twitter
1) Your periodic reminder of some things about your organization’s obsession with and fixation on testing for regression. a) Regression obsession reflects someone’s concern that developers are working so fast or carelessly that they’re prone to breaking stuff. That’s telling.
2) b) If (a) is true, regression-focused testing won’t solve that problem. If regression is a real risk, patient, careful, prudent development will more likely reduce it. c) Like all testing, testing for regression will miss some bugs. So see (b).
3) d) Regression testing is often considered solely in terms of running repeated automated checks, using limited oracles and limited variation, and therefore limited coverage. This approach will miss plenty of bugs (Beizer’s “Pesticide Paradox”) Increase coverage; vary checks.
4) e) People often point to low execution cost for automated regression checks—leaving out the cost of designing, developing, debugging, maintaining, describing, and interpreting them. f) Those costs tend to be lower when checks and product are built and maintained in parallel.
5) g) Regression checks are code, and yet they are often programmed by people with limited programming skill OR limited risk analysis skill. This results in even higher debugging and maintenance cost. Workaround: pair people with complementary skills, which aids learning.
6) h) Although low-level quality is essential, quality at the surface, at the human interface, where the people live and work is paramount. Don’t let fixation on regression checking displace testing that provides actual experience with and of the product.
7) i) Testing focused on other risks will find regression problems too. It will also help you to learn about kinds of change-related risk that you may not have considered. j) When regression checks produce red results, we pay attention. Beware of falling asleep on green ones.
8) k) Reflect on the assumption that green checks indicate “no problem”. They don’t. They indicate no problem *detected by the checks*. That’s VERY different. Checks—since they’re just code—are by nature oblivious and incurious.
9) l) An excellent test strategy needs to be diversified, product-specific, achievable, and risk-focused. So avoid fixation on regression risk. Diversify your regression testing, and diversify your testing to focus on other kinds of risk.
10) m) Last thing: “regress” means “going backwards”—the opposite of progress; things getting worse. Avoid saying “we’re going to do regression” or “we’re going to regress the build”. Instead try “we’re doing testing focused on regression-related risk” and sound like a pro.
