“How can I get into distributed systems?” - I get this question a lot, especially from people who read my distributed architecture concepts post - blog.pragmaticengineer.com/distributed-ar…
Here is the advice I give to people. A tread.
Here is the advice I give to people. A tread.
First, do your homework on the theory of distributed systems. It's an overloaded term. It usually means highly reliable systems that scale well to lots out users/requests. I have a few book recommendations - blog.pragmaticengineer.com/my-reading-lis…. Especially @martinkl 's book is gold.
@martinkl The Amazon Builder’s Library is also a good start. As part of doing your prep, look up all the terms you’re not familiar with - Wikipedia is a good resource to learn about each. CAP theorem, availability, strong/weak/eventual consistency, data durability and the lot.
@martinkl Then, look around and be (very) curious. You might already be working with dist systems, just ignoring them.
What API does your team/company have? What is the load? What is its infrastructure? Is it single- or multi DC? What happens, when it goes down? How reliable is it?
What API does your team/company have? What is the load? What is its infrastructure? Is it single- or multi DC? What happens, when it goes down? How reliable is it?
@martinkl Befriend the devs who build the largest systems at work. If you're lucky, you'll find high-load systems. This is what happened to me at Uber - I was a mobile engineer, but I was fascinated by the backend. I kept chatting up devs, reading our design docs and doing my homework.
@martinkl Distributed system or not, go oncall for a system that needs to be reliable. Build and tweak alerts, do postmortems when something breaks. Operating any reliable system is half of what distributed systems are about blog.pragmaticengineer.com/operating-a-hi…
@martinkl Your team doesn't have high-load systems? Befriend a team who does and offer to do oncall or shadow oncall pro bono. Here's a secret: barely anyone does this. A dev on my team did - good on him! He's become the oncall expert on our team as well (and later moved to the other team)
@martinkl If you don’t have that many people to learn from, reach out to local devs who build distributed systems. Search on your Linkedin 1st & 2nd degree connections. Reach out to these devs, offering to buy them lunch/cafe in return for sharing their experience. blog.pragmaticengineer.com/growth-hacks-c…
@martinkl If you don’t get to build these kinds of systems at work, you might decide to do some pet projects. It’s not that easy to spin-up a large scale system - but getting familiar with tools and technologies is the point with an exercise like this.
@martinkl Consider setting the goal to become deeply familiar with a distributed open-source tool, like Zookeeper, Kafka, Spark, Hadoop and many others (hadoopecosystemtable.github.io). Choose one, go deep & contribute to the project. Sounds hard? It is! This is why few people do it. Will you?
@martinkl If your company doesn’t have many opportunities, and you’re set on large systems work - you'll either need to create/get the opportunity to start something at scale or move to a place that already does this. The latter is usually a lot easier. But double check if #1 is an option.
@martinkl If you decide to move companies - startups give more freedom for to use new technologies. Talk to ones that have high current & projected growth. Larger companies at scale are also great - but harder to get into. FANG, Uber etc. Switching jobs is a whole other thread though.
@martinkl To get good at something, do a lot of it - to the point that you can teach it. So to get better at distributed systems, do your homework, then find ways to do more of this work. Get inspiration from others, find opportunities at work - or outside your current job. Good luck!
