Thread by @mipsytipsy

It blew my mind when I learned that most C-levels have a tighter relationship with their vendors than their own senior ICs.

Because engineers come and go, but vendors are forever. :mindblown:

https://twitter.com/MissAmyTobey/status/1301619668549918721

It does explain how vulnerable they are to the anti-instrumentation sales pitch.

The idea that you might not have to have anyone who deeply understands your systems? That you can pay them $$$$ and they will autoinstrument your code and tell you what to look at? So, so tempting.

WRT instrumentation: we can make it easier, we can gather a ton of stuff up automatically, we can write libraries to standardize and enable and more.

But auto-instrumentation is exactly as useful and as usable as auto-generated commenting for your code.

Which is because that is exactly what it is: a record of programmer intent.

Comments: "What I plan to do"
Instrumentation: "What I am doing"

Instrumentation (for observability) is simply commenting your code for interpretation at runtime.

(Instrumentation for metrics is a bit different, it often serves as a translation layer between code and low level systems statistics. But let's keep it simple.)

It's not a bad litmus test. If you don't have to do *some* instrumentation by hand, it probably isn't observability.

On the subject of senior engineers and their lack of fungibility: this gets more and more true the more senior they get.

When an engineer is growing from junior->intermediate->senior, we want them to become a reasonably well rounded senior engineer.

This doesn't always happen, sadly, but part of a manager's job is making sure you eat your vegetables.

That you don't just do the one or two things you enjoy and are good at over and over, but are exposed to various parts of the stack, and know enough not to be dangerous.

But once someone is solidly a senior engineer -- once you know what you like and what you don't, and are less of a danger to yourself and others -- then your path is in your hands.

And people tend to become more and more...specific...versions of themselves, as they grow.

Engineers who are 3-5 years out of college are way, WAY more fungible (on average) than engineers who are 10, 20, or 30 years removed.

I don't just mean when it comes to languages and technologies, either. The way you interact with your team+org is probably more important.