A question I hear a lot from engineering manager is, "How should I measure my software teams?" My initial answer, "I think you're asking the wrong question." Why? 👇
Many engineering leaders feel they should be measuring *something*, but it doesn't make sense to measure *anything* unless you know *why* you are measuring. This is the question to start with, "What do you care most about?"
Unfortunately, the answer to this is not easily distilled into a single number (or at least not that I've found). It's a subjective, biased and context-based answer which requires judgement
When you know what you care most about, then you might find a metric that might be an indicator. But be careful you don't use the metric as a short-hand for your goal
Because it's easy to make it a target... cue Goodhart's Law ("When a measure becomes a target, it ceases to be a good measure") en.wikipedia.org/wiki/Goodhart%… and people are *very* clever at gaming systems.
When I was studying #lean I remember a story of a call centre who wanted good customer service. They measured how long customers waited and made it a target to keep customers waiting no longer than 5 minutes on the telephone? Sounds good, right?
The number trended in the right direction. They managed to shorten the wait time for customers and over 3 months hit their target.
Customer waiting time fell, but overall customer satisfaction dropped dramatically. Why?
In order to answer quickly, customer service reps answered the telephone, but when they couldn't quickly resolve an issue, they had one of two responses 1) Escalate it or forward it to another colleague (not counted in initial waiting time), or to simply hang up 😱
I've been on the end of a telephone where it feels like this, bouncing from person to person, and sometimes simply being disconnected. The result? Higher customer frustration *and* more load on the overall system (I have to call back several times to get my query answered)
A single metric resulted in systematic degradation.
I had an interesting question for the talk I did today for @agilesgorg about my view on developer productivity like BlueOptima. I hadn't heard of it before but doing some reading now and 😱. Here's a quick thread with some initial thoughts
First there's a reminder about en.wikipedia.org/wiki/Goodhart%… (When a measure becomes a target, it ceases to be a good measure), or the #TOC version of this, "Tell me how you measure me and I'll tell you how I behave"
Leaders and managers need to think really, really, really hard about the metrics they use and the obtuse ways that people will game them. Also... engineers have an astounding talent for gaming the system, so beware... but back to the tool