Alex Xu Profile picture
Apr 26 โ€ข 6 tweets โ€ข 3 min read
How does Twitter work? Letโ€™s take a look at it from the architectural point of view before Elon takes it.

๐“๐ก๐ž ๐‹๐ข๐Ÿ๐ž ๐จ๐Ÿ ๐š ๐“๐ฐ๐ž๐ž๐ญ:
1๏ธโƒฃ A tweet comes in through the Write API.
2๏ธโƒฃ The Write API routes the request to the Fanout service.

#twitter #systemdesign
3๏ธโƒฃ The Fanout service does a lot of processing and stores them in the Redis cache.
4๏ธโƒฃ The Timeline service is used to find the Redis server that has the home timeline on it.
5๏ธโƒฃ A user pulls their home timeline through the Timeline service.
๐’๐ž๐š๐ซ๐œ๐ก & ๐ƒ๐ข๐ฌ๐œ๐จ๐ฏ๐ž๐ซ๐ฒ
๐Ÿ”น Ingester: annotates and tokenizes Tweets so the data can be indexed.
๐Ÿ”น Earlybird: stores search index.
๐Ÿ”น Blender: creates the search and discovery timelines.

๐๐ฎ๐ฌ๐ก ๐‚๐จ๐ฆ๐ฉ๐ฎ๐ญ๐ž
๐Ÿ”นHTTP push
๐Ÿ”นMobile push
Disclaimer: This article is based on the tech talk given by Twitter in 2013 (bit.ly/3vNfjRp). Even though many years have passed, itโ€™s still quite relevant. I redraw the diagram as the original diagram is difficult to read.
Over to you:
What are some of the biggest differences between Twitter and Facebook that might shape their system architectures?
If you found this thread helpful, follow me
@alexxubyte for more.

Retweet the first tweet to help more people to learn system design.

โ€ข โ€ข โ€ข

Missing some Tweet in this thread? You can try to force a refresh
ใ€€

Keep Current with Alex Xu

Alex Xu Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @alexxubyte

Apr 25
Popular interview question: What is the difference between ๐๐ซ๐จ๐œ๐ž๐ฌ๐ฌ and ๐“๐ก๐ซ๐ž๐š๐?

To better understand this question, letโ€™s first take a look at what is a Program.
A ๐๐ซ๐จ๐ ๐ซ๐š๐ฆ is an executable file containing a set of instructions and passively stored on disk. One program can have multiple processes. For example, the Chrome browser creates a different process for every single tab.
A ๐๐ซ๐จ๐œ๐ž๐ฌ๐ฌ means a program is in execution. When a program is loaded into the memory and becomes active, the program becomes a process. The process requires some essential resources such as registers, program counter, and stack.
Read 8 tweets
Apr 22
Interesting read: Software Architecture and Design InfoQ Trends Report โ€” April 2022 by @InfoQ Image
Key takeaways:
โ€œData plus architecture" is the idea that, more frequently, software architecture is adapting to consider data. This holistically includes data quality, data pipelines, and traceability to understand how data influenced decisions and AI models.
Innovative software architecture is facilitating data quality the way weโ€™ve improved code quality. Catching bad data early is as important as catching bugs early.
The practice of software architecture does not belong solely to people with the job title of architect.
Read 8 tweets
Apr 20
One picture is worth more than a thousand words. In this post, we will take a look at how to design ๐†๐จ๐จ๐ ๐ฅ๐ž ๐ƒ๐จ๐œ๐ฌ.

1๏ธโƒฃ Clients send document editing operations to the WebSocket Server.
2๏ธโƒฃ The real-time communication is handled by the WebSocket Server. Image
3๏ธโƒฃ Documents operations are persisted in the Message Queue.
4๏ธโƒฃ The File Operation Server consumes operations produced by clients and generates transformed operations using collaboration algorithms.
5๏ธโƒฃ Three types of data are stored: file metadata, file content, and operations. Image
One of the biggest challenges is real-time conflict resolution. Common algorithms include:

๐Ÿ”น Operational transformation (OT)
๐Ÿ”น Differential Synchronization (DS)
๐Ÿ”น Conflict-free replicated data type (CRDT)
Read 5 tweets
Apr 19
Deploying or upgrading services is risky. In this post, we explore risk mitigation strategies.

The diagram below illustrates the common ones. How to deploy services?
๐Œ๐ฎ๐ฅ๐ญ๐ข-๐’๐ž๐ซ๐ฏ๐ข๐œ๐ž ๐ƒ๐ž๐ฉ๐ฅ๐จ๐ฒ๐ฆ๐ž๐ง๐ญ
We deploy new changes to multiple services simultaneously. This approach is easy to implement. But since all the services are upgraded at the same time, it is hard to manage and test dependencies. Itโ€™s also hard to rollback safely. Image
๐๐ฅ๐ฎ๐ž-๐†๐ซ๐ž๐ž๐ง ๐ƒ๐ž๐ฉ๐ฅ๐จ๐ฒ๐ฆ๐ž๐ง๐ญ
With blue-green deployment, we have two identical environments: one is staging (blue) and the other is production (green). The staging environment is one version ahead of production. Image
Read 9 tweets
Apr 18
I quit my job at Twitter 3 years ago and put a huge burden of the mortgage on my wife. Bay area houses are expensive...

Now I make a living by writing a series of system design interview books with my co-author @sahnlam
Our books have 1000+ reviews on Amazon and are consistently ranked in the top 10 of Computer & Technology books.

I started to post on LinkedIn regularly and it passed 100,000 followers last Friday.

Iโ€™ve met a lot of like-minded individuals and love what I do.
Iโ€™ll be sharing my learning, thinking, or interview tips every week. If you think my posts will be useful, please like or share so they can reach a wider audience. Thanks for reading my story.
Read 4 tweets
Apr 13
How to design a ๐ฌ๐ž๐œ๐ฎ๐ซ๐ž web API access for your website?

When we open web API access to users, we need to make sure each API call is authenticated. This means the user must be who they claim to be.

In this post, we explore two common ways:
1. Token based authentication
2. HMAC (Hash-based Message Authentication Code) authentication

The diagram below illustrates how they work.

๐“๐จ๐ค๐ž๐ง ๐›๐š๐ฌ๐ž๐
Step 1 - the user enters their password into the client, and the client sends the password to the Authentication Server.
Step 2 - the Authentication Server authenticates the credentials and generates a token with an expiry time.

Steps 3 and 4 - now the client can send requests to access server resources with the token in the HTTP header. This access is valid until the token expires.
Read 9 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(