Thread by @michaelbolton

1) Documentation people and testers are like admins and secretaries. Companies came to think of them as an extravagance, and got rid of them in the name of "efficiency". But this is a completely bogus efficiency, because the cost of NOT having them is largely invisible.

2) Everybody is now swamped with doing the work that support people used to do, but that's invisible even to the people who are now performing that work. It's "just the way things (don't) get done around here". I notice this as I'm programming; most API documentation *sucks*.

3) Of course, when I'm programming (even for myself), my job is to make trouble go away; to get the problem solved. When something gets in the way of that, I'm disinclined to talk about it. "I'm a problem-solver!" So I'll buckle down and push on through. Gotta get it done!

4) When something finally does get done, my pain of having had to deal with problems dissipates, because "Hey, it works!" And who wants to talk about the problems any more? Plus, there's another issue: the details of those problems are BORING to anyone who isn't, or wasn't me.

5) The problems are even pretty boring to me. And I notice this, because... I've been logging them. In retropect, they're not *exciting*. But there is something kind of exciting about logging them: I find out how much time feels wasted because people aren't documenting stuff.

6) Working programmers and testers don't have the time to get their work done AND record what took them time, made things harder or slower. Plus they don't want to record obstacles because all the *capable* people *never* run into obstacles. (Well, they don't *say* they do.)

7) So what we have is this feedback loop: everything is horribly inefficient, but people are not prepared to say so, because inefficiency allows others to suspet some degree of incompetence, AND logging problems gets in the way of efficiency! This becomes the Secret Life of Tech.

8) Any kind of "finished" project is a ship in a bottle. No one sees the effort that went into it; no one sees the spilled paint or snapped hinges or tangled string or broken glass. (h/t Simon Shaffer and Harry Collins for those images). PLUS: the ship is delicate and fragile.

9) People look at the ship in the bottle, wonder for a moment how it got in there, and then shrug. There is *no way* for them to see the effort that went in, and they're also disinclined (for reasonable social reasons) to find out just how fragile the ship is. Don't poke it!

10) Part of the antidote to that, for testers, is to help our clients to become aware of aspects the Secret Life. The testing story cannot and must not be about the number of scripted tests that are passing, or the number of automated checks that are running. It's GOT to be more.

11) The testing story has to be about the actual, real status of the product, warts and all. Testers *must* focus on problems, because no one else wants to do that — but bad problems threaten the value of the product. Better to experience them and learn about them in the lab.

12) But for that story to have a warrant, we must tell a story about how we tested, and how we observed problems. But we must also tell a story about important testing not yet done, and what we won't test at all unless things change. Important untested stuff is where risk lurks.

13) So those strands in the larger testing story need to get wrapped around another strand: what made testing harder, slower, less efficient, less valuable—and what we need or recommend to make some aspects of testing faster, easier, more efficient, or even unnecesssary.

14) Wait, what? When would testing be unnecessary? Testing is much less necessary when you know THIS well enough that more testing probably won't reveal more problems — which means you can now test THAT, or FOR THAT; places or conditions where deeper or more subtle problems lurk.

15) It seems that many testers are being pressured to Get Stuff Done with goals that don't help very much: showing that everything is okay. Even better if we can WRITE PROGRAMS to show that everything is okay—which means a big investment in time and effort to miss the point.

16) Testing and running automated checks never show that the product *works*. They can only show that the product CAN work. That's not the point of testing. We test to find out where the product DOESN'T work, and where it MIGHT NOT work, so that those problems can get addressed.

17) What's worse, the bulk of the time and effort needed to demonstrate that the product CAN work gets swept under the rug because of things mentioned earlier. What's the matter with the testers? They take all that time, and can't even write a program to show our product works!

18) There are LOTS of possible reasons for that. One, of course, is that the product doesn't work, which really messes with writing a program to show that it works. Another is that the product is not understood comprehensively by anyone; the product is like Rashomon; fragmented.

19) Even when there is a reasonably coherent product and story,—clean, shared mental models, good access to its elements—it takes time and effort to learn a product well enough to do deep testing on it. Just like the invisible work of admins, that's the invisible work of testers.

20) So one important part of testing, little discussed by the certification mills, is the complex social, political, and economic task of making invisible testing work more visible and legible to our clients, to keep them informed about risk. developsense.com/blog/2018/02/h…

21) Another cool service that testers *could* provide is either informing or outright producing documentation about the product. In a sense, skilled testers do (or can and should be doing) that anyway; that's the Honest Manual Writer Heuristic. (developsense.com/blog/2016/05/t…).

22) When someone asks "So why have testers anyway?" a typical answer, provided hesitantly with a rising tone is "To... provide... information?" Sure, but what kind? Let's be specific: valuable information about the actual status of the product, what it does, and problems in it.

23) The "what it does" and "problems in it" part could be really valuable, because companies and groups (and open source projects!) are often oblivious to THEIR clients' needs to know about how the product works. This is utterly clear from looking at most current documentation.

24) Testing is fundamentally learning about the product, thinking critically about it, exploring it, experimenting with it, *experiencing* it, and reporting on those experiences—especially the painful parts—to help companies become aware of the pain they might inflict on clients.

25) Some of that learning work can be aided by machinery, but applying tools to testing in a powerful way requires us to learn about and study the product deeply. That learning process requires us to have *and develop* good models of the product AND of testing itself.

26) If you're still with me so far, a word from our sponsor: Rapid Software Testing is all about this stuff. Please spread the word: RST Explored runs April 12-15 in time zones friendly to Europe, the UK, the Middle East, and India. eventbrite.ca/e/rapid-softwa…