Thread by @mipsytipsy

It's our all day planning kickoff at @honeycombio, and star customers @ceejbot and @shriganeshram are here from Eaze to talk about how observability helped them with their "exciting" monolith rewrite and migration. 😍 #starstruck

"At first we took our .Net monolith and spun it out into node microservices, as people tend to do when they have no idea why they are doing microservices...That went well"

lol yesssss

"We spent the first month doing nothing but investigation, to understand our old system. We instrumented and got spans working, and found bugs like -- a loop around a network call that caused it to fetch the same data 100x from redis.

With honeycomb, it was INSTANTLY obvious."

lol they literally made a spreadsheet using honeycomb data to decide what to work on first 🤓

@ceejbot "I look forward to not having to write any more load testing code. It's just useless; you HAVE to use production data." 🙌 PREACH 🙌

"You cannot write a complex working system. So we have to start by rewriting simple working parts of the system."

"I don't get how you can even try to run a distributed system without observability. They exist in a constant state of failure."

"You just HAVE to have very fine grained detail. ... You HAVE to have high cardinality. Honeycomb gives the detail of logs without the limitations of metrics." damn @ceejbot spitting truth faster than I can scribe it

(Not scribing the list of tech they tried and failed or ripped out bc I do not gratuitously shit on competitors) 😇

also, multiple shout outs to @mranney for inspiring @ceejbot and raising her bar for how amazing data visualizations can be... 🖤🌈 (miss you Matt!)

"Observability gives me confidence in the business.". @shriganeshram describing the demoralizing effects of cascading downtimes.

"It was a tech debt doom spiral... you can't make forward progress and all development becomes extremely costly" @ceejbot

"We can't afford observability is a myth... We can't afford to NOT have observability." @ceejbot now describing how honeycomb helps them understand and control costs, and lets them hire fewer headcount.

@lizthegrey asks, "what would you tell people in a similar position?"

@ceejbot "it is SO costly to grind your team down to that point. It is so much cheaper to prevent this pain of not understanding a breaking system than to recover from it. SO, so much cheaper. Augggghhhh."

"What I learned from @mranney is how much more powerful metrics can be than just simple system metrics and "how long does it take." You can actually start to understand your system in rich and meaningful and complex ways that map to what your users really care about."

lol we have so many questions for these poor saints, we are never going to let them leave ☺️💜💙

(thank you two soooooo much!! Such an awesome fucking story)