,
9 tweets,
3 min read
Read on Twitter
i'm feeling inspired seeing all these tweets and comments flooding out of #monitorama.
so i'm going to tweet some factoids and common misunderstandings throughout the day as they come to me (or as i notice people committing said sins) #o11yfacts incoming!
so i'm going to tweet some factoids and common misunderstandings throughout the day as they come to me (or as i notice people committing said sins) #o11yfacts incoming!
#o11yfacts 1: there are two different meanings of the word 'metrics'.
There are small-m "metrics", a generic collective noun for statistical data. And there are statsd "Metrics", which are a specific data type consisting of a single number with n tags appended to it.
There are small-m "metrics", a generic collective noun for statistical data. And there are statsd "Metrics", which are a specific data type consisting of a single number with n tags appended to it.
Tags are a way of adding dimensions to the number so they can be sorted, grouped and described more usefully.
Statsd Metrics can be one of a small number of types, e.g. counter, gauge, histogram, summary, timing, etc, which inform the aggregator how to process or bucket them.
Statsd Metrics can be one of a small number of types, e.g. counter, gauge, histogram, summary, timing, etc, which inform the aggregator how to process or bucket them.
For a long time statsd and its descendants (wavefront, signalfx, datadog, etc) have been the dominant game in town when it comes to monitoring.
This means a lot of people have the Metrics data model lodged firmly in their heads, and assume that Metrics and metrics are the same.
This means a lot of people have the Metrics data model lodged firmly in their heads, and assume that Metrics and metrics are the same.
Metrics are fast, cheap, small, have a fixed schema, and are a great fit for tsdbs -- you can pre-aggregate before they even reach the server, then downsample and lose resolution over time so you retain long term data without increasing your storage footprint or cost.
But Metrics have severe limitations, too.
You might fire off hundreds of Metrics over the course of a single request, yet they are disconnected from one another; there is no way of guaranteeing that any two of them describe the same request or were true at the same time.
You might fire off hundreds of Metrics over the course of a single request, yet they are disconnected from one another; there is no way of guaranteeing that any two of them describe the same request or were true at the same time.
If you take advantages of the properties that make them cheap, you quickly find yourself in the realm of nonsense numbers that make no god damn sense, because they're averages of averages (of averages).
And statsd descendants only handle low cardinality dimensions.
And statsd descendants only handle low cardinality dimensions.
Therefore, Metrics-based systems are primarily usable for monitoring known-unknowns or describing the health of the system in aggregate over time.
They are terrible for debugging scenarios, or trying to understand user experience, or unknown-unknowns. Just about impossible.
They are terrible for debugging scenarios, or trying to understand user experience, or unknown-unknowns. Just about impossible.
Lots of people use metrics/Metrics interchangeably. I never say it unless I'm referring to statsd-style Metrics, to avoid confusion.
As you investigate observability and adopt newer paradigms for complex systems, you may want to familiarize yourself with this difference. 🍻
As you investigate observability and adopt newer paradigms for complex systems, you may want to familiarize yourself with this difference. 🍻
