I was just editing the o11y book chapter on build vs buy and ROI, and this sentence jumped out at me:
"High-performing organizations use great tools."
It's true, right? Behold all the FAANG engineers who leave their cushy perches and are shocked by the amount of tooling they had come to take for granted. It's almost like having to learn to engineer all over again
Big companies know how critical good tooling is, and pay for it.
I'm going to say two very contradictory things, both of which are true:
1) Tools are getting better and better, and you should try to keep up
2) Switching tooling is hard, and you should only do it when the gain is ~an order of magnitude better than what you've got.
Engineering cycles are likely the scarcest resource in your organization, so it's truly astonishing how wasteful most places are with them.
I guess it's easier to hire another eng than it is to thoughtfully pay down tech debt and reclaim cycles. ¯\_(ツ)_/¯
There are some tools where spread and variety is all fine and good. Take editors -- to each her own, it doesn't matter.
There are other tools where consolidation and consistency of use are so critical that deviation would derail the team, like git. And plenty in between.
Then there are the tools in the middle, where some conventions may be required, but mostly intra-team. Like ... pagerduty, or feature flags, or the whole suite of debugging, observability, and monitoring tools.
Do some teams perform worse because they lack the tools to do better, or do they lack the tools to do better because they are worse teams?
My vote is generally the former. But even for engineering orgs that don't fall down the BYO hole, tooling can be a major source of drag.
Consider just a few of these tooling questions:
* new tool is LOADS better for some teams, only minorly so for others. should you make them (needlessly, to their mind) switch?
* you just finished rolling out the new tool, and you already regret your decision. UGH. what to do?
* you've never bothered to standardize in the past, but it's becoming a nightmare. there are at least eleven solutions running internally ... that you know of.
* in 👆 situation, one engineer is a committer to the open source solution and says they will quit if another is chosen
* your team LOVES one solution -- unanimously! the problem is, you're a RoR shop and this is a java solution. should you maintain all these servers and configs *just* for this tool?
* your teams LOVES one solution, but you have to stop using it for compliance reasons.
* you have standardized on one tool, but there are many workflows, and each has their passionate proponents. also you have no decision-making authority, only influence. what to do?
* you have one tool, but 50 different versions in use. people SAY they will upgrade, but never do.
* the bootstrapping guide has become a 30 page monstrosity. someone needs to figure out how to automate it, image the setup and manage the laptops. you're too small to hire an IT person. which lucky engineer gets the short stick?
You get the point (and I'm sure you all have hair-raising stories of your own!).
The point is, tooling decisions are incredibly important. Especially considering how ...let's say "lightweight".. the evaluation process often was in the first place. 🙃
There's no rule book. Sometimes variance is fine, sometimes not. Sometimes it's worth bearing down on folks, often it's not.
It takes a lot of good judgment, experience, and political/interpersonal skills to guide these decisions to a good outcome.
I'll just say -- if you asked me, "what's the best way to sabotage an engineering org, to frustrate them and force them to spin their wheels, hate each other and stop producing much of anything at all?"
I'd reply, "sell them on some bad tools." 😈
• • •
Missing some Tweet in this thread? You can try to
force a refresh
You don't owe it to your employer to fix all the ways they are fucked up. Before going to battle, ask yourself:
* how much power do I have here?
* is the problem within my domain of responsibility or influence?
* who are my allies?
* do I have a reasonable chance of success?
and also: are they worth it? Is your employer fundamentally worth you staying and fighting? Is their product a net good for the world? Are your leaders decent, ethical people who care a lot?
If so, sure, pick some battles. See what happens. ☺️
Ah! This is a very good point. Good recruiters are outnumbered by bad ones, which are indistinguishable from spam. And yes, the more you put out the more you'll get.
Here we are, now going on the fourth straight month of headlines all about how a record number of people are quitting their jobs.
There's a lot of pain behind that statistic, but also a strident, activated edge to labor that feels unlike anything seen in my lifetime.
I am *all for* more people quitting their jobs. I am *all for* employers needing to compete for employees by treating them better, increasing their wages, and offering more flexibility and support.
Most people in our industry stay at jobs they don't love, far too long.
So here's a piece of advice that I find myself giving over and over again, to senior folks who are daunted by the prospect of having to go out and search for the right role, the right team, the right company ... it's like looking for a needle in a haystack, right? 😰
This is 100% true. What always jumps out at me is that there's always "the tracing expert(s)" that everybody goes to for help on the rare occasions when they need a trace. Fluency rarely transcends the few to reach the many.
I don't think it's this so much as it is that ... tracing is *inherently* a niche use case. A tracing-first approach to observability turns the world on its head for no good reason.
What most people want is the ability to slice and dice their requests,
Ostrich effect: ignoring an obvious (negative) situation
IKEA effect: The tendency for people to place a disproportionately high value on objects that they partially assembled themselves, such as furniture from IKEA, regardless of the quality of the end product
and some delightful ones that I failed to work in:
Zeigarnik effect: That uncompleted or interrupted tasks are remembered better than completed ones.
Tachypsychia: When time perceived by the individual either lengthens, making events appear to slow down, or contracts
the answer is no, we can't; there were no off the shelf columnar dbs, let alone any with flexible schemas or the rest of our wishlist.
if we had shoved it in an existing data store we would have looked and felt just like every other monitoring tool. same perf, same tradeoffs.
the VC's were right too, though; we did almost doom ourselves. 🙃 it took us nearly a year before we could even really start signing up users or BEGIN working on the product. it was 2.5-3 years til we found PMF.
meanwhile our seed investors gave up on us long before that.