,
11 tweets,
3 min read
Read on Twitter
ABSOFUCKINGLUTELY. Operating code in production is important, but is just the beginning. The goal is to extend the same tools and practices backwards to cover the full life cycle of your code.
Consider the honeycomb user maturity model:
Consider the honeycomb user maturity model:
0 -- when they come to us, teams usually have no real time access to prod (delayed by minutes), and no single source of truth with ad hoc exploring. To debug a problem they jump from alerts, to dashboards, to logs and other sources; correlating manually/visually, rinse&repeat.
They also have limited or no instrumentation from the perspective of their code. So that's step one.
1 -- instrument their code with beelines, so they gather the right data and bundle it correctly. Now they have real time access to prod as experienced by their users.
1 -- instrument their code with beelines, so they gather the right data and bundle it correctly. Now they have real time access to prod as experienced by their users.
Now you can put a software engineer on call without being unreasonable.
So we're done, right??
So we're done, right??
Stage one is about incident investigation, the reactive-exploratory stage where you have lots of burning questions about system failures that can't be answered any other way. The team is too overwhelmed to do anything but fight fires.
Stage two is proactive-exploratory: making it so you can ship software faster, with more confidence. Bringing everyone up to the level of the best debugger in every problem area. Practicing observability-driven development with feature flags, and "testing" in prod.
Stage three is proactive-opportunistic -- looking outward for ways to be done to better serve your customers by extending o11y to other teams. Fine-tuning processes around shipping software, using o11y for auto-remediation. Usually comes with huge increases in productivity.
And stage four is strategic-committed. This is when we see companies have an ingrained habit of "observe, deploy, observe", and have instilled a firm culture of end-to-end software lifecycle ownership. Every surface is instrumented, unpredictable events are no longer scary.
Again, this is our attempt to group the users we have observed at various points of the journey from beginner to mastery. The ordering is sometimes mixed up, e.g. sometimes instrument their build pipeline before ingesting db queries, but the start and end poles are consistent.
(I started it off kind of confusingly, apologies, decided to switch the focus from us to them around stage 1. lol)
Check out some of our case studies at honeycomb.io/customers/. Intercom is a great example of a company that adopted us very early on, and generally "doing it right". (They independently arrived at many of the same conclusions as us.)
