making instrumentation/telemetry just "how we work" vs. some bolt-on process, telephone game, as-a-PM-I-can-see user story craziness.
I see this with super successful customers at @Amplitude_HQ . It is just how they work.
How do you do it? 1/n
First, you have to get the people who 1) understand the decision domain, 2) understand the customer domain, 3) understand the product surface area, and 4) understand "data" ... in one place.
Sometimes you get lucky and that is one or two people. Sometimes not.
Gotta do it 2/n
Usable data starts with domain knowledge!
It is so tempting to just instrument abstract clicks or page views and hope to clean it up later.
But instrumenting clean, understandable, domain relevant events from the start is SO MUCH BETTER.
Then...3/n
You have to get away from imagining you're going to figure out every question in advance, or "spec" the perfect dashboard.
Sure, keep reference questions in mind
But focus on "things that happen that we care about"
The nouns, verbs, adjectives, adverbs
That skill...4/n
...is the foundation for making instrumentation just part of each and every new interaction.
Instead of an extra ticket, you just ask yourself with each new design/flow/interface
"what is actually happening here?"
and instrument as part of the "story"
5/n
The code takes m/secs to write. With the right testing env you can validate it works in minutes.
The muscle you have to build is figuring out how to integrate this into your workflow.
That's what makes this feel heavyweight at first.
Tools help 6/n
Tools like @iteratively get you out of spreadsheets and into a repo-like environment.
I know this from my adtech days...as versatile as they are, spreadsheets are terrible for event tracking plans.
So tool up 7/n
Finally, this is a habit like any other.
Before you start it feels heavyweight. There is a big temptation to decouple this from "the work".
Resist that and start thinking about instrumentation is core as writing tests. Bake it into your "definition of done" by default 8/end
• • •
Missing some Tweet in this thread? You can try to
force a refresh
But there's one huge trap that I see teams fall into.
Start with the Why not the Way
Visualizing work is not the goal. ___ is the goal.
What do I mean?
1/n: Imagine if you emptied out all of your messy drawers just for fun. Well..
2/n: You would have succeeded in making a big mess and reminding yourself how messy you are, and how much you like collecting old subway cards, but you wouldn't have really achieved anything.
Now say you....
3/n: Started by committing to a powerful mission of making it easier to find things. You spend valuable time every day checking multiple drawers.
Or committing to a more public display of keepsakes and caring for your things better?
was asked recently how I would go about "benchmarking self-service analytics performance".
Some thoughts:
1/n: You can't *just* look at the experience of the end-user. There are many humans involved in making self-service work. Their experience matters.
2/n: A great example is telemetry/instrumentation.
Someone has to figure out how to capture that data. What is the developer's instrumentation experience? What is the experience of deciding which events to track? How fragile is the process?
That's part of performance.
3/n: You can't measure performance by focusing solely on accessibility to the data or insights. Or even the timeliness of the data.
At the end of the day the goal is better decisions and business outcomes.
Is stop trying to optimize for developers "being busy"
...prepping work to give them
...doing small group discovery upstream from their work
..."topping up" every sprint (or quarter, or whatever)
The hardest part? ... 1/n
...often the push to keep people "loaded up" comes from the engineering org itself. People want to feel useful. People want to have something to do
Output is rewarded.
If there is "no work", that is the product managers fault. People get grumpy
So how do you address this? 2/n
For many teams, it may mean having a list of small things people are passionate about. Things that preserve optionality, and that are relatively quick and low risk
When you need slack, you can draw from this list
This helps for the ppl who don't care for doing discovery
3/n
1/n: First, the worst possible way to sell what you want to do is to preach about lowering WIP! No one cares! Lowering WIP is a Way not a Why.
People may nod, but deep inside they are saying "WTF is she talking about?". So...
2/n: You'll need to position this in a way that people will universally care about it. Like:
"I want to increase the sustainable flow of value creation through this system. Right now a rough way to gauge that is to look at ___, and I want to double that w/o hiring more people"
3/n: Yes, all measures can be abused. But you'll need some way to divert their attention from the Way, and give you the leeway.
To prevent abuse, add a...
"without adversely impacting X,Y, Z, A, B, C" with a mix of qual and quant