The #1 sign that someone does not have observability is this: they lean on alarms for ⭐️EVERYTHING.⭐️
They cannot debug or introspect their systems any other way, so they abuse the FUCK out of monitoring checks and alarms.
Sorry if that was a little harsh. Flashbacks. Shudder.
It is NOT EASY trying to cobble together an understanding of a complex system using checks, alarms, dashboards, and your intuition. Sweetheart, I *know*. You do your best
But your systems are getting exponentially more complex. Every dimension is high cardinality with a growing set -- services, apis, instances, containers, endpoints, storage...
It's all growing a lot faster than *that* exponential curve. You cannot subject a team to this barrage of alerts and thresholds.
Or if like they are on call and trying to repro something for a particular user.