How to scale a website to support millions of users? We will explain this step-by-step.
The diagram below illustrates the evolution of a simplified eCommerce website. It goes from a monolithic design on one single server, to a service-oriented/microservice architecture.
Suppose we have two services: inventory service (handles product descriptions and inventory management) and user service (handles user information, registration, login, etc.).
Step 1 - With the growth of the user base, one single application server cannot handle the traffic anymore. We put the application server and the database server into two separate servers.
Step 2 - The business continues to grow, and a single application server is no longer enough. So we deploy a cluster of application servers.
Step 3 - Now the incoming requests have to be routed to multiple application servers, how can we ensure each application server gets an even load? The load balancer handles this nicely.
Step 4 - With the business continuing to grow, the database might become the bottleneck. To mitigate this, we separate reads and writes in a way that frequent read queries go to read replicas. With this setup, the throughput for the database writes can be greatly increased.
Step 5 - Suppose the business continues to grow. One single database cannot handle the load on both the inventory table and user table. We have a few options:
1. Vertical partition. Adding more power (CPU, RAM, etc.) to the database server. It has a hard limit. 2. Horizontal partition by adding more database servers. 3. Adding a caching layer to offload read requests.
Step 6 - Now we can modularize the functions into different services. The architecture becomes service-oriented / microservice.
Question: what else do we need to support an e-commerce website at Amazon’s scale?
• • •
Missing some Tweet in this thread? You can try to
force a refresh
Twitter has enforced very strict rate limiting. Some people cannot even see their own tweets.
Rate limiting is a very important yet often overlooked topic. Let's use this opportunity to take a look at what it is and the most popular algorithms.
A thread.
#RateLimitExceeded
What is rate limiting? Rate limiting controls the rate at which users or services can access a resource. Here are some examples:
- A user can send a message no more than 2 per second
- One can create a maximum of 10 accounts per day from the same IP address
Fixed Window Counter
The algorithm divides the timeline into fixed-size time windows and assigns a counter for each window. Each request increments the counter by some value. Once the counter reaches the threshold, subsequent requests are blocked until the new time window begins
/1 What is the difference between “pull” and “push” payments?
The diagram below shows how the pull and push payments work.
/2 🔹 When we swipe a credit/debit card at a merchant, it is a pull payment, where the money is sent from the cardholder to the merchant. The merchant pulls money from the cardholder’s account, and the cardholder approves the transaction.
/3 🔹 With Visa Direct or Mastercard Send, the push payments enable merchant, corporate, and government disbursements.
Step 1: The merchant initiates the push payment through a digital channel. It can be a mobile phone or a bank branch etc.
/2 The Netflix Engineering team selects a variety of databases to empower streaming at scale.
Relational databases: Netflix chooses MySql for billing transactions, subscriptions, taxes, etc. They use CockroachDB to support a multi-region active-active architecture.
/3 Columnar databases: Netflix primarily uses them for analytics purposes. They utilize Redshift and Druid for structured data storage, Spark and data pipeline processing, and Tableau for data visualization.
Communication between different software systems can be established using either RPC (Remote Procedure Call) or RESTful (Representational State Transfer) protocols, which allow multiple systems to work together in distributed computing.
/2 The two protocols differ mainly in their design philosophy. RPC enables calling remote procedures on a server as if they were local procedures, while RESTful applications are resource-based and interact with these resources via HTTP methods.
/3 When choosing between RPC and RESTful, consider your application's needs.