1/n
Increasing the no of users on-chain requires faster execution times, higher security, & cheaper txn fees.
#Sharding is one solution that claims to scale blockchains to process up to a million txns per second.
What's sharding? How does it work?
Let's find out 🧵👇
2/n
In non-sharded blockchains, each node is responsible for processing every transaction and storing the entire state.
Even if there are 1000 nodes (⬆️ nodes = ⬆️ security), the TPS of the network is limited to the processing power of one node.
This creates a bottleneck.
3/n
Sharding is one of the most viable solutions to this problem.
It focuses on splitting the blockchain into smaller pieces called "shards."
Each shard can process a portion of the total transaction processing load, increasing the network's processing power.
4/n
The nodes in the shards execute transactions and store states limited to that shard.
Increasing the chance of Sybil attacks (by making it easy to execute) as the validator set is split into shards.
5/n
One challenge in this approach is that you can scale up to a certain capacity depending on the beacon chain capacity.
Although adding infinite shards is possible, you also have to scale the beacon chain to adjust that load.
The solution is Quadratic Sharding.
6/n
What is it?
If nodes include the beacon chain, then each shard will be able to process 4x more transactions & beacon chain will be able to maintain 4x more shards.
Throughput across the system will increase by 4 × 4 = 16
7/n
A node is responsible for three things-
- Processing transactions
- Storing the transactions data
- Relaying transactions to other nodes in the network.
Sharding partitions these three operations using the network, state & transaction sharding.
8/n
Network sharding: It handles the way the nodes are grouped into shards.
This can be used to optimize communication since message propagation inside a shard is much faster than propagation to the entire network.
9/n
State Sharding: The state of the blockchain is split across shards.
Nodes in a particular shard only process & relay txns limited to the state of that shard.
This linearly reduces storage & compute requirements and introduces problems of DA and cross-shard txns.
10/n
Transaction Sharding: Sharding of the compute power of a #blockchain network.
This is easy because every node has the whole state of the blockchain.
The only need is to ensure that shards don’t update the same state concurrently.
@zilliqa blockchain uses this.
11/n
Certain challenges arise due to sharding designs:
1. Cross-chain communication
2. State Validity/Data Validity
3. Data Availability
How are these tackled? Let's take a look. 👇
12/n
Cross-shard communication issue: If smart contracts are hosted on different shards & they want to send a message from one shard to another, it's not possible for nodes in one shard to complete these txns.
There are two approaches to solving cross-shard communication.
13/n
1. Synchronous: The blocks in both shards for the cross-shard txns are produced in sync, & validators collaborate on executing such txns. e.g., #Merge blocks technique.
14/n
2. Asynchronous: This approach executes txns asynchronously where it's possible for one shard to verify txns in another shard. e.g., the Beacon chain technique used in @Polkadot .
15/n
State validity problem:
A shard cannot confirm the current #shard state they are interacting with without downloading and validating the entire shard history.
This problem doesn't exist in non-sharded blockchains.
16/n
Validator rotation:
Rotating validators reduces the probability of malicious #validator behavior, i.e., colluding to produce a fork or invalid block. Allowing cross-shard txs only between neighbouring shards.
Nodes can verify blocks in their shards & neighbors as well.
17/n
Fishermen:
Whenever a block header is published to the beacon chain, there is a challenging period for honest validators to prove that a block is invalid.
Succinct proofs:
Cryptographic proofs could prove a computation's validity, i.e., block production based on txns.
18/n
There are 2 types of nodes:
Full nodes: Downloads all blocks and validates every txns.
Light nodes: Downloads only block headers.
Validators in the shard are full nodes for that shard, & other participants in the system, including the #Beacon chain, operate as light nodes
19/n
Data availability problem:
Light nodes don't know when a new block is produced in a shard because all data in that block has already been published.
The dilemma is if block producers don't release all data, light nodes can't detect malicious txns hidden in that block.
20/n
Proof-of-Custody: Have notaries in shards on a rotating basis, downloading the blocks & confirming that they can do so.
21/n
Erasure codes: Light nodes verify data availability of block by downloading random chunks & verifying using #Merkle root for validity. @ethereum & Polkadot have solutions based on this.
22/n
Benefits of sharding
-High #TPS: Limited nodes execute transactions rather than all nodes.
-Reduced compute & storage needs: State size and processing load of shards < the whole blockchain.
-#Decentralization: Node requirements decrease, making it easy to run and maintain.
23/n
Multiple blockchain networks have introduced sharding to their scaling plans.
Some of them are:
~ #Zilliqa
~ #Polkadot
~ @NEARProtocol
~ #Ethereum 2.0
~ @harmonyprotocol
~ @ElrondNetwork
~ #Shardeum
Let's look at some of their design choices. 👇
24/n
In Zilliqa, shards handle processing but not storage i.e., the transactions are processed parallelly across shards while every shard holds a full blockchain state.
👇
25/n
The same state is shared across all shards, so different techniques are used to make sure that shards don’t update the same state concurrently.
26/n
@NEARProtocol sharding has a single blockchain design rather than beacon chain & shard chains.
Each block has all shard txns & can alter all shard states.
Nodes only maintain the state for their shard.
Txns are split into chunks, & one chunk represents one shard.
27/n
@Polkadot uses #Parachains to achieve distributed database sharding.
Parachains attach to the security, a relay chain that provides rather than providing their own.
A key feature of parachains is that the computations they perform are inherently independent.
28/n
@ethereum sharding design is based on shard & beacon chains. It uses a receipt mechanism for cross-chain communication.
There is also a proposition to create 64 shards to scale Eth up to 100K txns per second.
It is yet to launch after the most awaited merge happens.
29/n
@harmonyprotocol has sharding on 3 dimensions, i.e., state, transactions, and network.
It has a beacon and shard chain design.
Crosslinks are used to post valid block headers to the beacon chain.
It allows cross-shard transactions as well.
30/n
@ElrondNetwork uses a novel state sharding technique called Adaptive state sharding.
It combines state + transaction + network sharding & allows to dynamically change no of shards based on nodes available and network usage to achieve efficient sharding.
31/n
Shardeum is an EVM-based sharded blockchain that uses dynamic state sharding.
It has unique features such as transaction-level consensus, which eliminates atomic processing complexities faced by block-level #consensus designs.
32/32
Sharding has huge potential to scale blockchains to serve the growing demand.
It's currently in development across chains; the first blockchain to solve for the complex design will attract users & accrue higher value
Follow @GravityX_Cap for more in-depth research.
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.