Thread by @GeePawHill

I spoze the historic and ongoing inability/unwillingness of the software trade to grasp and adopt test-driven development (TDD) is one of the most frustrating & demoralizing events of my forty-two years as a professional geek.

I believe there are several related factors in play, ranging in abstraction level from pressures of global ieconmics to mistakes in local human interaction. Studying this large-scale failure, even while having some small-scale successes, underlies much of my work on change.

Because, while the overall world of geekery uses TDD approximately not at all, I have had many personal successes, using it, teaching it, and coaching it. Reconciling this seeming contradiction is hard.

My current thesis is that it comes down to a combination of lousy timing and even worse handling.

The stretch from '95 to '05 was, frankly, a really lousy time to introduce *any* change in second- and third-tier geek thought and practice.

(By "tier", here, I mean some rough combination of skillset, mindset, and experience. By "second" and "third" I mean to refer to the leaders, de facto and de jure, of technical teams, people who are themselves still technical on a daily basis.)

The second technical revolution was going on. The first -- cheap physical computing -- was slowing down but still active. The second -- essentially infinite free distribution -- was rolling out like a house afire.

This created a truly extraordinary marketplace. I'd go so far as to say "unique", though I'm not being rigorous, and I won't claim perfect knowledge of all prior markets.

Making software was ridiculously cheap, and ridiculously profitable, and demand was insatiable, and the pressure it put on supply was unbearable.

That supply, the geek trade, was dancing as fast as it could, in the year 2000.

We were churning out product, making billions of dollars of profit, and we were doubling our geek census every four or five years.

To heighten the pressure? The demand was so high, we discovered that we could do this all rather badly and *still* make more money than we lost.

And who was doing this? Who was keeping all these wobbly balls in the air, simultaneously geeking 40 hours and managing 40 hours and planning 40 hours every week?

Right. The second and third tier of geeks, the leaders, some young, some pushing forty, the people who knew enough geekery to keep it all afloat.

You can do a lot when you're dancing as fast as you can. You can code. You can ship. You can run a team. You can double-check everything they do. You can re-direct bad product ideas. You can hire and fire.

The one thing you *can't* do, though, when you're dancing as fast as you can: you can't learn a whole new kind of dance.

TDD is not an after-market bolt-on to the geekery that came before it. It is a set of practices based in a sophisticated theory about what parts of programming are hard, what parts easy, about what parts break, and what parts don't.

To take the simplest possible practical impact of the TDD dance, the designs you make with TDD are different -- sometimes radically -- from the designs you make without it. Just because of TDD.

There's more than that, too, but that, alone, is a huge deal.

So there's this lousy timing. The very people we needed to turn on to this different dance were the people who were utterly and completely swamped. Anything we had to teach them had to pay off in a matter of *days*.

TDD was not the only great idea emerging in that stretch. The design patterns movement. Extreme Programming itself. Late-stage OO theory, all of these were happening. None of them get much attention or mindshare today, and for much the same reason.

But wait. Act before midnight tonight, and we'll not only send you the lousy timing, but look, our operators are standing by, and we'll throw in this *exclusive*: really bad handling.

Duplicate bridge is a weird world. You play bridge, the card game. And you play against a whole big tournament of other people. But it's not fair, right, what if my hand is crummy, and the other pair gets a good one?

Well, with duplicate, we fixed that. See, every table gets the exact same deck. And we don't compare your performance against the people at your table. We compare your performance against all the people who were dealt the same hand you were.

It's one thing to get dealt a lousy hand. It's another thing to play whatever hand you were dealt in a lousy way.

It was a lousy time to introduce a new sophisticated idea. But that was our hand. And we didn't play it very well.

To begin with, tho we liked the practice of TDD, and were gradually learning how best to use it, our theory about what made it work was not very strong yet. Introducing a practice w/o any idea of what makes it work is a recipe for failure. We pushed too hard, too soon.

We were desperate for mindshare. We wanted people to do this, and we wanted them to pay attention to it. This led us to a couple of very challenging problems.

TDD's great, you think, and you want everybody to do it. But without some serious underlying sense of how what we were doing actually worked, it gets very hard to say what the "it" really is.

There are a lot of things that resemble TDD. Test-after Developers writing tests. Non-developers writing screen-scraping scripts. Full or large-scale integration tests. There's even more.

None of them are TDD, but they sound like they are.

And this lack of understanding led to not just one follow-on problem, but a second major one, and then even more problems after that.

In the early '70s, the Doonsebury cartoon had Lacey and Dick Davenport, a wealthy patrician couple. And Dick was fuming one day, if I recollect it was about the Nixon whitehouse. And he said, "That's the trouble with the Republican Party, they let just anybody in."

Without a deeper understanding of how TDD works, and because of our desperate desire for mindshare, anything resembling TDD, regardless of whether it relied on the same underlying mechanisms or not, was good enough.

We let just anybody in.

And I'm not speaking here of formal certification, or review boards, or any of that. I'm talking about culture, not rules. I wouldn't "enforce" such an idea as TDD even if I could.

But when you let just anybody in, especially in a market as insane as the geekery market, you get some follow-on results that aren't good.

You get rote. You get bad pedagogy. You get pedagogues who don't even care, who are in it for the income. You get significant failures. You get empty slogans repeated over and over and over ad nauseum.

And the worst part is when you "win".

When you make the cover of an airline magazine, with no coherent understanding of what TDD is or why it works, you get top-down pressure to make line geeks do a thing they don't understand and, especially, don't believe in.

We got all of that.

And as a result? We got a lot of implacable foes, a lot of awful-sounding and true failure reports, and a lot of "influencers" decrying our pratice.

And now, when we have a much richer understanding of what it is, of how it works, of *when* it works, and of why it works, well, we're too little too late.

Lousy timing, lousy handling. Both of these are flavors of rush rush rush. The timing was so bad because the would-be recipients were rush rush rush. The handling was so bad because the would-be TDD'ers were rush rush rush.

Rushing, as hard as you can, directly at your goal, regardless of your hand, your timing, your context, it seems right, mete and just, and it seems *so* important!

But it's nearly always a flaw in your change plan. It was a terrible flaw in ours. And now, to an arbitrary error epsilon, the uptake of TDD overall in the trade -- a model I regard as the most significant advance in geekery of my entire career -- is zero.

Thanks for hanging in this long. I didn't mean this to be such a downer, and I feel a little silly asking you to take action as a result. Just the same, tho, if my thinking about change interests you, do me one or two favors:

1) Take my seminar in "Leading Technical Change". 12 people, 4 sessions, 2 hours, across 2 weeks.

One topic:

Getting to Real Change in the Real World.

geepawhill.org/courses/leadin…

2) Subscribe. It's free, it's spam-free, and it includes full-text and audio of these threads, right to your inbox.

geepawhill.org/subscribe

I'mo have a cold beer and listen to some tunes. Y'all have a good night and a better tomorrow.