James Cowling Profile picture
Jan 16, 2025 11 tweets 3 min read Read on X
How to be a Principal Engineer/Senior Principal Engineer/Senior Architect/fancy-sounding-title Engineer, a thread:

1. You're evaluated on how much more the company succeeds because you're there, not the lines of code you wrote. If you can unblock someone, do that. If you need to kill a two year project that's not going anywhere, do that. Do what is right, not what makes you look good.
2. Your job is the strategy stuff and the dirty work stuff. All the cool stuff in the middle is for everyone else. You're not too senior to carry a pager or respond to outages, this keeps you in touch with how things are going. Image
3. If you get too caught up in the day-to-day (the how) to step back and think about whether you're going in the right direction (the why and the what) the team will eventually go adrift and people will lose motivation. You have to proactively own this.
4. Engineers are often bad at strategy because it involves unknowns and engineers like to be confident and exact. Getting good at long term strategy and dealing with unknowns is the #1 skill to develop:
medium.com/@jamesacowling…
5. If you spend all day in your little strategy bubble you will probably not have the credibility from the wider engineering team, and if you don't have credibility it's hard to get things done. It helps a lot to have spent serious time at the company building things and continuing to do so.
6. You're on the solutions team, not the complaining team. If you don't like something then go fix it or go speak to someone about getting it fixed. Sometimes you also just need to let things go.

You also need empathy for company leadership. They are probably not a bunch of idiots, they're likely just juggling a different set of priorities. If they really are idiots then quit and go work for a better company.
7. Pick your battles. Not every engineering decision needs to be perfect. Sometimes people need to make small mistakes and learn from them. Sometimes you were wrong and it wasn't a mistake after all. Holding people accountable is way better than micromanaging them.
8. You have to make choices! Choices are decisions where there is no single correct obvious outcome. You have to do it and then you have to own it when it's not perfect. This is usually better than doing nothing.
9. Don't try to hide behind data driven decision making. Sometimes there's just no data to support what has to be an intuitive decision. Just because it's measurable doesn't mean it's the most important factor. Use data to reflect on the decision afterwards and learn from it.
10. Do your best to be made redundant. Mentor folks, try to help people grow, give junior engineers the maximum autonomy and responsibility they can handle. Once someone else can do your job then you can go take on bigger challenges.
11. Don't be an asshole. The tech industry is small and everyone knows the people they trust and who they'd want to work with again. This is the iterative prisoner's dilemma and the optimal strategy (career wise and as a human being) is just to be a good person and try to do the right thing.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with James Cowling

James Cowling Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @jamesacowling

Apr 10
Time for a big systems advice thread!

In distributed systems there's no magic "push everything to prod at once" button. Every service gets pushed independently and nodes within a service get updated incrementally. If you mess up forwards/backwards compatibility you can fail irrecoverably.

So how to avoid this?

1/5: Decouple data and code changes. Never push out a release that changes how data is stored at the same time as the code that uses this new data. If there's a bug and you need to roll back to the old version of your code it won't be able to handle the new data in the new format. Instead push out a release that first changes the data in a way that’s compatible with both the old and new code (e.g., optional fields etc), when that’s stable push out the new code that uses it, then when that’s stable you can change the data to remove backwards compatibility. This is known as a “migration” in the database world and yes it’s annoying, but yes you need to do it.
2/5: Don’t change two services at once. If service A talks to service B, you can’t just add a new API to both of them and push them out. What if someone pushes A but not B? What if there’s a bug in B that needs to be rolled back? Just like with data changes, API changes need to be made in a forwards and backwards compatible way. Engineers forget to do this all the time.
3/5: Only allow one version “step” to exist in prod at any time. It’s common to have most of your nodes at version 5 but a few are still at version 4 because they haven’t finished migrating. Never ever allow someone to push to version 6 while version 4 is still running. Otherwise it’s too hard for engineers to reason about which version is “stable” when making multi-step migrations. You need monitoring and alerting for this plus protections agains corner cases, e.g., if a node was offline and then came back online after missing a code push.
Read 6 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(