Thread by @mipsytipsy

1. The only lens you should need for debugging your code is that of your own instrumentation.

2. Never let code into production unless you can answer the question: how will I know if this breaks or doesn't work?

https://twitter.com/windlessuser/status/1100222816773988357

3. Instrument your code while pretending you're a serverless dev, and you'll do all the right things.

4. Instrumentation isn't just for "at scale", it is the only way to inspect what your code *actually does* when it meets infra and users. It will improve your life today.

5. Magic auto-instrumentation will only ever catch the dumbest, most generic problems. It doesn't have to be *hard* or tedious to instrument, but someone, somewhere needs to roughly understand your code when instrumenting.

6. Arbitrarily wide events are the foundational data type for debugging, metrics and aggregates and traces can all be derived from them. Context is everything. Flexible schemas a must.

7. Structure. Your. Fucking. Data.

8. You must be able to run read-time aggregation and computation across raw events in order to ask new questions without shipping new code.

9. Sample -- don't pre-aggregate -- to manage costs. Practice dynamic sampling, retain fewer common events and more of uncommon.

10. Delete all your paging alerts. Rely on the big three and a few end-to-end checks or other high level proxies for user pain. Don't rely on page bombs for debugging. Only page if users are impacted, and work towards a world where everything can fail without impacting users.

There you go, ten commandments of instrumentation sprung forth ad hoc-edly on my walk home. Who wants to argue with me, or replace or amend any of them? The floor is open... 🌈💖🤗