Tweet

@sahnlam

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

More from @alexxubyte

Alex Xu

@alexxubyte

Mar 7

Popular interview question: how to diagnose a mysterious process that’s taking too much CPU, memory, IO, etc?

The diagram below illustrates helpful tools in a Linux system.

🔹‘vmstat’ - reports information about processes, memory, paging, block IO, traps, and CPU activity.

🔹‘iostat’ - reports CPU and input/output statistics of the system.

🔹‘netstat’ - displays statistical data related to IP, TCP, UDP, and ICMP protocols.

🔹‘lsof’ - lists open files of the current system.

🔹‘pidstat’ - monitors the utilization of system resources by all or specified processes, including CPU, memory, device IO, task switching, threads, etc.

Read 4 tweets

Alex Xu

@alexxubyte

Mar 4

Design stock exchange. Let’s trace the life of an order through various components in the diagram to see how the pieces fit together.

First, we follow the order through the trading flow. This is the critical path with strict latency requirements. Everything has to happen fast in the flow:

Step 1: A client places an order via the broker’s web or mobile app.

Step 2: The broker sends the order to the exchange.

Step 3: The order enters the exchange through the client gateway. The client gateway performs basic gatekeeping functions such as input validation, rate limiting, authentication, normalization, etc. The client gateway then forwards the order to the order manager.

Read 6 tweets

Alex Xu

@alexxubyte

Feb 25

You probably heard about 𝐒𝐖𝐈𝐅𝐓. What is SWIFT? What role does it play in cross-border payments? Let's take a look.

The Society for Worldwide Interbank Financial Telecommunication (SWIFT) is the main secure 𝐦𝐞𝐬𝐬𝐚𝐠𝐢𝐧𝐠 𝐬𝐲𝐬𝐭𝐞𝐦 that links the world’s banks. 1/9

The Belgium-based system is run by its member banks and handles millions of payment messages per day. The diagram below illustrates how payment messages are transmitted from Bank A (in New York) to Bank B (in London). 2/9

Step 1: Bank A sends a message with transfer details to Regional Processor A in New York. The destination is Bank B. 3/9

Read 9 tweets

Alex Xu

@alexxubyte

Feb 24

In modern architecture, systems are broken up into small and independent building blocks with well-defined interfaces between them. Message queues provide communication and coordination for those building blocks. Today, let’s discuss at-most once, at-least once, and exactly once.

𝐀𝐭-𝐦𝐨𝐬𝐭 𝐨𝐧𝐜𝐞
As the name suggests, at-most once means a message will be delivered not more than once. Messages may be lost but are not redelivered. This is how at-most once delivery works at the high level.

Use cases: It is suitable for use cases like monitoring metrics, where a small amount of data loss is acceptable.

𝐀𝐭-𝐥𝐞𝐚𝐬𝐭 𝐨𝐧𝐜𝐞
With this data delivery semantic, it’s acceptable to deliver a message more than once, but no message should be lost.

Read 8 tweets

Alex Xu

@alexxubyte

Feb 23

In many large-scale applications, data is divided into partitions that can be accessed separately. There are two typical strategies for partitioning data.

🔹 Vertical partitioning: it means some columns are moved to new tables. Each table contains the same number of rows but fewer columns (see diagram below).

Horizontal partitioning (often called sharding): divides a table into multiple smaller tables. Each table is a separate data store, and it contains the same number of columns, but fewer rows.

Horizontal partitioning is widely used so let’s take a closer look

Read 9 tweets

Alex Xu

@alexxubyte

Feb 18

A really cool technique that’s commonly used in object storage such as S3 to improve durability is called 𝐄𝐫𝐚𝐬𝐮𝐫𝐞 𝐂𝐨𝐝𝐢𝐧𝐠. Let’s take a look at how it works. 1/7

Erasure coding deals with data durability differently from replication. It chunks data into smaller pieces and creates parities for redundancy. In the event of failures, we can use chunk data and parities to reconstruct the data. 4 + 2 erasure coding is shown in Figure 1. 2/7

1️⃣ Data is broken up into four even-sized data chunks d1, d2, d3, and d4.

2️⃣ The mathematical formula is used to calculate the parities p1 and p2. To give a much simplified example, p1 = d1 + 2*d2 - d3 + 4*d4 and p2 = -d1 + 5*d2 + d3 - 3*d4. 3/7

Read 7 tweets

Share this page!

Alex Xu

People who liked this thread also liked...

Try unrolling a thread yourself!

More from @alexxubyte

Alex Xu

Alex Xu

Alex Xu

Alex Xu

Alex Xu

Alex Xu

Did Thread Reader help you today?

Don't want to be a Premium member but still want to support us?

Like this author's thread?