Thread by @mipsytipsy

... also, this thing where you have multiple sources of truth (metrics, dashboards, traces, logs) and hop around correlating by hand.... is MADNESS.

This must end.

https://twitter.com/betsythemuffin/status/1085281348678496257

This will only end when folks adopt arbitrarily wide events as the single source of truth. Because traces are just a visual format and metrics/rollups can be derived from events, but it doesn't work in reverse.

Tired of hand-correlating and spending a fortune? Try @honeycombio

I find it pretty annoying and cynical that our competitors seem to know this and are racing to catch up behind the scenes, while muddying the water and misleading their users with their messaging.

Honeycomb: we may be self interested, but our brand is not lying to you. 🙄

Speaking of which, time for another round of:

🐝 Observability is NOT a synonym for telemetry
🐝 Observability is NOT a grab bag of metrics, traces, and logs
🐝 Observability is a measure of how well you can reason about the insides of your systems, by observing from the outside

Observability requires you to be able to ask new questions -- any question, arbitrary questions -- without shipping new code.

For that, you need computation/aggregation at read time & access to raw events. No preaggregation, that locks you in to the questions you aggregated on.

Dashboards and preaggregates are very useful for other things! But for debugging software in distributed systems, you need it to be oriented around the request as it hops across your system.

This is how your user experiences it, so this is what matters.