Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
Charity Majors Profile picture
@mipsytipsy
Sep 14, 2020 • 11 tweets • 3 min read • Read on X
Scrolly
Updated definition:

📉 Monitoring is for running and understanding other people's code (aka "your infrastructure")

📈 Observability is for running and understanding *your* code -- the code you write, change and ship every day; the code that solves your core business problems.
Questions monitoring tools (like datadog, signalfx) can answer:

* When will my disk fill up?
* Am I running out of capacity in $(cluster)?
* Did the % free memory drop after my last deploy?
* What is the avg, 90th, 99th percentile latency per service?
Questions observability tools (like honeycomb, lightstep) can answer:

* What (1..many) things do all the errors in that spike have in common?
* How many exports per second is $app doing, and how large are they, and how does this compare to the average export size in kB?
* Break down by app and sort by export size: what are your top 3 export users, and what is the sum of their total throughput compared to the overall throughput?
* Are the errors evenly distributed across workers, AZs, instance types, software versions, build_id versions, shards?
* Are the timeouts happening for all our users, or only the test users, or only our top users by write volume, etc?
* For all of the deliveries that failed over a specified time period, what are the top three reasons they failed, and what % of failures were from a single app?
Running infrastructure means running black boxes. You may have some insight into them (god i hope so) but you don't have the ability to tweak their instrumentation, and you certainly aren't shipping code changes every day.

Infrastructure code gets package-upgraded, rarely.
And when it comes to monitoring and understanding your infrastructure, metrics-based monitoring tools that let you understand performance in aggregate are the tool for the job.

Esp when workloads are high throughput with little differentiation (routers, etc) metrics are king.
When it comes to aligning developer perspective with user experience to provide core business value, though: event-based observability tools are the only way to get at the information you need.

You need the flexibility and precision of a scalpel, not an axe.
To see an expert yet beginner-friendly (and entertaining!) intro to observability for business problems, check out this talk from @seebails -- observe2020.io/2020/03/chris-…
And to continue my killjoy track record of stiffly caring about technical definitions for technical terms, if you'd like to read more, please read my three year history of observability in the software domain. aka how we got here and where we're going:

thenewstack.io/observability-…
annnd -- should you be in the mood to build an o11y tool in-house, or want to argue with me about why datadog and signalfx and their ilk are definitely not observability tools (or about what constitutes an o11y tool), do read this:

honeycomb.io/blog/so-you-wa…

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Charity Majors

Charity Majors Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @mipsytipsy

Charity Majors Profile picture
Mar 18
I devoured the recap @martinfowler posted from the deer valley summit. Loved it.

But the notes suggest we may be replicating a perennial blind spot in software engineering: treating code like the outcome, and production like an afterthought.

honeycomb.io/blog/productio…
The notes from the summit are here,

A key area of discussion was, "where does the rigor go?" based on @chadfowler's extremely influential post on the topic.

Production! The rigor must go to production! Or what are we even doing here?thoughtworks.com/about-us/event…
Formal methods and test suites are like flight simulators.

Flight simulators are genuinely impressive. Airlines use them, pilots log real hours, they catch real failure modes. Nobody's saying skip the simulator.

But a pilot who has only flown simulators is not a pilot.
Read 8 tweets
Charity Majors Profile picture
Sep 21, 2024
I've written a few threads in recent memory on skip levels, and tbqh, some of these responses I've gotten have thrown me for quite a loop.

"You should talk to people who don't report to you directly" -- I did not think this would be a controversial statement!

Whuf 🙈
Replies came from all over the place:

* what's a skip level?

* why would I *want* to talk to my boss's boss? Sounds boring, what would even we talk about?

* why *should* I talk to my reports' reports? I don't want to confuse them, and anyway there are just too many of them
The most troubling ones came from middle managers themselves:

* "my manager should only have to talk to my reports if she doesn't trust me"

* "skips are disempowering to managers and confusing to my reports"

* "what if they criticize me behind my back & I can't defend myself?"
Read 15 tweets
Charity Majors Profile picture
Sep 2, 2024
I woke up this am, scanned Twitter from bed, and spent an hour debating whether I could stomach the energy to respond to the latest breathless fatwa from Paul Graham.

I fell asleep again before deciding; just as well, because @clairevo said it all more nicely than I would have.
(Is that all I have to say? No, dammit, I guess it is not.)

This is so everything about PG in a nutshell, and why I find him so heartbreakingly frustrating.

The guy is brilliant, and a genius communicator. He's seen more and done more than I ever will, times a thousand.
And he is so, so, so consistently blinkered in certain predictable ways. As a former fundamentalist, my reference point for this sort of conduct is mostly religious.

And YC has always struck me less like an investment vehicle, much more like a cult dedicated to founder worship.
Read 30 tweets
Charity Majors Profile picture
Aug 30, 2024
I would soften this a bit. Managers are *not* the only ones who can transform the culture of an org, but their buy-in and support are fundamental.

Good managers are channelers. They use their power to filter, amplify and leverage voices around the org to shape company culture.
Important context: that post was quote tweeting this one.

Because I have also seen designers come in saying lovely things about transformation and user centricity, and end up wasting unthinkable quantities of organizational energy and time.

If you're a manager, and you have a boot camp grad designer who comes in the door wanting to transform your org, and you let them, you are committing professional malpractice.

The way you earn the right to transform is by executing consistently, and transforming incrementally.
Read 5 tweets
Charity Majors Profile picture
Aug 28, 2024
Another way of looking at this is: if the real product of any software engineering team is shared understanding,

if the speed of sense-making is a core limiting factor on the speed of delivery,

then strong, futureproof engineering teams are 🌸continuous learning machines🌸
(by "futureproof" I mean "true 5y from now whether AI is writing 0% or 100% our lines of code)

And you know what's a great continuous e2e test of your team's prowess at learning and sensemaking?

1, regularly injecting fresh junior talent
2, composing teams of a range of levels
"Is it safe to ask questions" is a low fucking bar. Better: is it normal to ask questions, is it an expected contribution from every person at every level? Does everyone get a chance to explain and talk through their work?

Are great questioners celebrated? Rewarded?
Read 8 tweets
Charity Majors Profile picture
Aug 21, 2024
The advance of LLMs and other AI tools is a rare opportunity to radically upend the way we talk and think about software development, and change our industry for the better.

The way we have traditionally talked about software centers on writing code, solving technical problems.
LLMs challenge this -- in a way that can feel scary and disorienting. If the robots are coming for our life's work, what crumbs will be left for you and me?

But I would argue that this has always been a misrepresentation of the work, one which confuses the trees for the forest.
Something I have been noodling on is, how to describe software development in a way that is both a) true today, and b) relatively futureproof, meaning still true 5 years from now if the optimists have won and most code is no longer written by humans.

I have two propositions.
Read 24 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

Send Email!

:(