,
12 tweets,
3 min read
Read on Twitter
In 2003, Cem Kaner introduced me to the idea that software testing is a social science. To believe that testing is purely and solely a technical examination is to miss the point: that we’re evaluating whether the software will work well enough for the people who will use it.
2. In 2013, Harry Collins made the point sharper: since computers and software are machines, they should be checked to make sure there are no missing teeth and the gears mesh nicely. But because of human purposes, that’s only the beginning of what we must do to test really well.
3. Computers are not members of society; they are not and cannot be, in the foreseeable future, socialized as people are, and so they fail to behave as reasonable social agents would. The humans who interact with them must “repair” these deficiencies, often in real time. 3/4
4. So, to test well, we must apply complex social judgements to decide whether people will tolerate the amount of repair they have to apply to the interaction—which means *far* more than checking whether the wheels spin right; far more than fact-checking in the CI/CD build. 4/5
5. Yet these days, I see many testing roles turning into CI/CD build roles. Hey, reliable builds are pretty important. But they don’t mark the end of actual testing, challenging the product and looking for problems. They’re the point at which experiential testing can begin. 5/6
6. Analysis by @jamesmarcusbach and me: testing can be put in four frames. Intention: team-related test activity in which we identify *and question* what we want to build. Discipline: programmer-focused test activity in which we build *and examine what we’re building critically*.
7. Preparation: team-focused test activity in which we get ready for *and question* the things we need to do fast, effective, and appropriately deep testing (including asking “has some of our testing already been deep enough relative to certain risks?”). 7/8
8. The fourth frame: Realization, focused on two ideas thanks to a double meaning: have we realized—accomplished—our intention? And have we realized—recognized—all of the important problem in the product that we reasonably could? These four frames can be applied at all levels.
9. Consider applying those four frames whether we’re looking at the entire landscape of the project; to a single-line-of-code change; to a user story; to a sprint; to our test strategy. Try considering them in a traditional, DevOps, Scrumbannerfall, or Agile context. 9/10
10. It seems to me lately that overfocus on building the product has led us to apply a lot of focus to Discipline and Preparation, and a little to Intention. I worry that we’re not paying nearly enough attention to Realization, and not linking it sufficiently to Intention. 10/11
11. Realization is where we examine, explore, experiment, and gain experience with the product. In the Realization frame, a big part of the job is to refocus on Intention and ask if we’re solving people’s problems—and whether we’re introducing new problems that they must repair.
12. We must remain vigilant, lest we release products that are technically correct, yet annoy, frustrate, confuse, or slow down people—or that cause loss, harm, or diminished value. Checking that the build goes smoothly is NOT doing that; it’s only preparation for doing it. 12/12
