After spending a few days using Prometheus, I'm a little stumped as to why it's so popular. I think my expectations were out of whack with the amount of buzz surrounding it.
I get that there isn't much else in terms of OSS monitoring solutions (*shivers at the days of using Graphite and Grafana*)...
But I would much rather throw piles of cash at Datadog or some other vendor than run my own monitoring infrastructure and use Prometheus based on my experience. For some reason people look at you weird when you say things like that though.
I was just a little underwhelmed with Prom—setting it up, running it, scraping metrics, this weird Pushgateway thing for batch and serverless jobs. It's cumbersome.
Having to use exporters everywhere is wildly clunky. I have a bunch of intermediate components to get metrics to Prometheus which just adds a bunch of failure points and things I have to manage.
Not to be overly critical, I know it's a high quality piece of software, builds on Google's monitoring best practices, etc. etc.
But the whole pull model just seems...dated? A relic of Borgmon? I'm not sure. It just doesn't seem all that well-suited to containers and serverless worlds and better for static infrastructures (yes I know Google's been using containers forever). I don't get why it's popular.
What makes this thing the "cloud-native" monitoring tool?
I prefer the push model. I don't want to have to worry about scaling my Prometheus instances, making sure they're scraping the right endpoints, running exporters to expose metrics or gateways to push them.
I just want to dump data to stdout—logs, metrics, doesn't matter as long as it's structured. Shovel it into a scalable pipeline like Kafka, Kinesis, Cloud Pub/Sub, route it wherever it needs to go. This is the observability pipeline model I describe here: bravenewgeek.com/the-observabil…
If Google's Stackdriver had a marginally better UX it would do most of what I need, and I can build push mechanisms around it. Sadly, it's janky as hell.
*IF* I had to run my own monitoring stack, I'm not sure I'd use Prometheus. I might just use an ELK stack fed from a push pipeline.
Still, I'd rather pay someone to do it for me. Someone who can actually invest the money to build a highly usable product and allow me to not worry about running ancillary infrastructure, i.e. blog.realkinetic.com/stop-wasting-y…
