I'm going to log here my journey learning more about distributed systems. #LearnInPublic
First of all, let's start with an introduction to that theme!
In a single-computer, concurrency is known as shared-memory concurrency as multiple threads can access the same memory space.
First of all, let's start with an introduction to that theme!
In a single-computer, concurrency is known as shared-memory concurrency as multiple threads can access the same memory space.
In a distributed system, concurrency still exists as different computers can execute programs in parallel. However, shared memory is not typically present. Each computer in a distributed system has its own operating system and address space, and nodes communicate with each other.
They communicate by sending messages over a network. Databases can be thought of as shared memory, but with a different data model. Each computer in a distributed system is called a node, which can be any type of communicating computing device.
Another reason for using distributed systems is for better performance.
If a service has users all over the world and they all have to access a single node, it can lead to slow performance. By placing nodes in multiple locations, we can route users to nearby nodes.
If a service has users all over the world and they all have to access a single node, it can lead to slow performance. By placing nodes in multiple locations, we can route users to nearby nodes.
Some large-scale tasks are too big or slow to be performed on a single computer. For example, the Large Hadron Collider has a worldwide computing infrastructure with 1 million CPU cores and 1 exabyte of storage.
Why make a system distributed? TL;DR
> It’s already distributed:
When you send a message to your friend, we have at least two devices involved.
> For better reliability:
Even if one node fails, the system as a whole keeps
functioning
> It’s already distributed:
When you send a message to your friend, we have at least two devices involved.
> For better reliability:
Even if one node fails, the system as a whole keeps
functioning
> For better performance:
Get data from a nearby node rather than one halfway
round the world
> To solve bigger problems:
Huge amounts of data, can’t fit on a single machine
Get data from a nearby node rather than one halfway
round the world
> To solve bigger problems:
Huge amounts of data, can’t fit on a single machine
However, there are also downsides to distributed systems, because things can go wrong, and the
system needs to deal with such faults. The network may fail, leaving the nodes unable to communicate.
In distributed systems, nodes may crash, run slow, or misbehave in some way.
system needs to deal with such faults. The network may fail, leaving the nodes unable to communicate.
In distributed systems, nodes may crash, run slow, or misbehave in some way.
To handle these situations, we need to detect when a node has crashed and another node take over. However, network and node failures can happen without warning, making detection difficult. In a single computer, if a component fails, the computer will likely crash.
In a distributed system, we need to design for resilience and handle failures gracefully.
In a single computer, the software doesn't need to be written to explicitly handle faulty RAM. However, in a distributed system, we often want to tolerate partial failures.
In a single computer, the software doesn't need to be written to explicitly handle faulty RAM. However, in a distributed system, we often want to tolerate partial failures.
If a component in a system stops working, it is called a fault. Distributed systems try to provide fault tolerance, meaning the system continues to function despite the fault.
Handling faults is what makes distributed computing fundamentally different and often more challenging than programming a single computer.
Why NOT make a system distributed? TL;DR
Communication: may fail (and we might not even know it
has failed).
Processes may: crash (and we might not know).
Fault tolerance: we want the system as a whole to continue working, even when some parts are faulty.
Communication: may fail (and we might not even know it
has failed).
Processes may: crash (and we might not know).
Fault tolerance: we want the system as a whole to continue working, even when some parts are faulty.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
