What is Adaptive State Sharding?🧵
This is a Power Tool created by the #MultiversX team to run their protocol in a dynamically scalable way
This is going to be a long & technical one!🙉
For those wanting to further understand this cool mechanism💪
#cryptocurrency #technical
This is a Power Tool created by the #MultiversX team to run their protocol in a dynamically scalable way
This is going to be a long & technical one!🙉
For those wanting to further understand this cool mechanism💪
#cryptocurrency #technical
Content: 🤩
💠What is it?
💠Types of Sharding
💠MVX's Approach
💠How it is automated?
💠Technical Mechanisms Used
💠Cross-Shard Communication
💠Static vs Adaptive State Sharding
💠Sharding Vulnerabilities
💠Comparison to Other ⛓️
💠Great Thread & Resources
💠What is it?
💠Types of Sharding
💠MVX's Approach
💠How it is automated?
💠Technical Mechanisms Used
💠Cross-Shard Communication
💠Static vs Adaptive State Sharding
💠Sharding Vulnerabilities
💠Comparison to Other ⛓️
💠Great Thread & Resources
💠What does it mean?🤔
It partitions the transactions and wallet addresses into multiple shards, handled in parallel by a random selection of different participating validators
This makes it easier to update a protocol as different technologies or user needs arise🧠
It partitions the transactions and wallet addresses into multiple shards, handled in parallel by a random selection of different participating validators
This makes it easier to update a protocol as different technologies or user needs arise🧠
It brings:
>Higher Transaction speeds to the network
&
>Reduced Gas fees 😮
>Higher Transaction speeds to the network
&
>Reduced Gas fees 😮
💠How does it work?
To add shards: The nodes in a specific shard must use a binary tree to split wallet addresses in a hierarchical way
From there, a shard with a lot of volume can split in half leaving half nodes and addresses in one state and the other half in the new one
To add shards: The nodes in a specific shard must use a binary tree to split wallet addresses in a hierarchical way
From there, a shard with a lot of volume can split in half leaving half nodes and addresses in one state and the other half in the new one
To remove a shard, the network must merge 2 shards together. Which entails the nodes combining their collective data for both shards, and sharing one shard in a new state
💠Types of Sharding:
>State= Dividing the entire blockchain data (state) into shards (most difficult)
>Transaction= Process of dividing tx among shards
>Network= Process of grouping nodes into shards
MVX utilises all 3 in its Sharding mechanism 👏
>State= Dividing the entire blockchain data (state) into shards (most difficult)
>Transaction= Process of dividing tx among shards
>Network= Process of grouping nodes into shards
MVX utilises all 3 in its Sharding mechanism 👏
💠#MultiversX's approach
>Scalability with Availability= Able to scale up and down without affecting downtimes or minimising it for updates
>Dispatching & Instant Traceability= Destination shard of a tx should be deterministic
>Efficiency & Adaptability= Shards should be balanced
>Scalability with Availability= Able to scale up and down without affecting downtimes or minimising it for updates
>Dispatching & Instant Traceability= Destination shard of a tx should be deterministic
>Efficiency & Adaptability= Shards should be balanced
💠Latency
This is the cost required by nodes to retrieve the new state once their shard address is modified
The cost is reduced using a binary tree to determine the number of shards in an epoch
This is the cost required by nodes to retrieve the new state once their shard address is modified
The cost is reduced using a binary tree to determine the number of shards in an epoch
Therefore there is no cost to split shards as each node keeps 1/2 of previous address space for new space
Secondly, the "state redundancy mechanism" means that there is reduced cost in merging shards as the state in the sibling nodes are retained by each node for the new state
Secondly, the "state redundancy mechanism" means that there is reduced cost in merging shards as the state in the sibling nodes are retained by each node for the new state
💠How MVX Automates this 🤯
This may look complicated but what it is showing is the blockchain decides to add or remove shards based on a deviation of both:
>Total Number of Nodes
&
>Average Number of Transactions per Block
Automatically executed at the end of an epoch
This may look complicated but what it is showing is the blockchain decides to add or remove shards based on a deviation of both:
>Total Number of Nodes
&
>Average Number of Transactions per Block
Automatically executed at the end of an epoch
For a Split function:
>Total Number of Nodes > (Optimal per shard+allowed deviation)
&
>Average tx per block > Threshold number of tx per block
Merge function: is conducted only if the number of nodes < a set threshold below the optimal number of nodes in a shard
>Total Number of Nodes > (Optimal per shard+allowed deviation)
&
>Average tx per block > Threshold number of tx per block
Merge function: is conducted only if the number of nodes < a set threshold below the optimal number of nodes in a shard
It is worth pointing out here that in the Total Number of Nodes calculation the protocol includes:
>Eligible Validators
>Nodes on the waiting list
>Newly added nodes in the node pool
>Eligible Validators
>Nodes on the waiting list
>Newly added nodes in the node pool
💠Take a break You Deserve It to get through all that!🤠
This is very complex and I will go more in detail soon
So for now enjoy this puppy playing around🤗
This is very complex and I will go more in detail soon
So for now enjoy this puppy playing around🤗
💠Dispatching Mechanism
In order to further increase throughput (tps) & reduce cross-shard communications (gas) MVX has added a Dispatching Mechanism
This automates the determination of a tx between any shards involved. Avoiding any confusion
In order to further increase throughput (tps) & reduce cross-shard communications (gas) MVX has added a Dispatching Mechanism
This automates the determination of a tx between any shards involved. Avoiding any confusion
💠Sharding Tree Structure
This shows a Binary Tree structure for adding shards
It is balanced up until the last row where there are 5 shards, 2 of which will contain 1/2 of the data the other 3 have & therefore 1/2 of fee rewards
MVX mitigates this by...
This shows a Binary Tree structure for adding shards
It is balanced up until the last row where there are 5 shards, 2 of which will contain 1/2 of the data the other 3 have & therefore 1/2 of fee rewards
MVX mitigates this by...
...having a 1/3 of shard nodes randomly redistributed each epoch & balancing the distribution of nodes according to the tree level
💠Reducing Confusion 🙊
To reduce confusion when adding and removing shards MVX has developed a mechanism to calculate instant traceability
When a wallet is created it is put in an originating shard & is identifiable by its last 'n' bits of account addresses
To reduce confusion when adding and removing shards MVX has developed a mechanism to calculate instant traceability
When a wallet is created it is put in an originating shard & is identifiable by its last 'n' bits of account addresses
ID's given to shards are incremental. So if 2 merge the highest number one is taken away
Using this function the network can determine a wallets parent or sibling shard. Allowing for the wallet to remain in the same shard throughout scaling
This is important for Shard Redundancy
Using this function the network can determine a wallets parent or sibling shard. Allowing for the wallet to remain in the same shard throughout scaling
This is important for Shard Redundancy
💠Shard Redundancy
For Blockchain, State sharding is vulnerable to shard failure either through:
>All nodes in the shard being offline (e.g. maybe same location at 3am)
or
>Failure to reach consensus (more than 1/3 of nodes not responding)
MVX forces shards to hold siblings data
For Blockchain, State sharding is vulnerable to shard failure either through:
>All nodes in the shard being offline (e.g. maybe same location at 3am)
or
>Failure to reach consensus (more than 1/3 of nodes not responding)
MVX forces shards to hold siblings data
A sibling shard is the other shard to the nodes in a shard that is created when a shard is split
It can help:
>Nodes to validate that sibling shard in unlikely times
&
>It reduces communication need for any merge that may happen in the future. As they already have all the data
It can help:
>Nodes to validate that sibling shard in unlikely times
&
>It reduces communication need for any merge that may happen in the future. As they already have all the data
Are you still with me? 👀
Don't worry we are getting close to the end!
Don't worry we are getting close to the end!
💠Context Switching
This is when 1/3 of nodes are redistributed at the start of each epoch
It is the most inefficient & costly mechanism in the MVX protocol, as the redistributed node must then resync the state, blockchain & txs with the eligible nodes in the new shard
This is when 1/3 of nodes are redistributed at the start of each epoch
It is the most inefficient & costly mechanism in the MVX protocol, as the redistributed node must then resync the state, blockchain & txs with the eligible nodes in the new shard
But it provides a level of security that no other blockchain has implemented (to my knowledge)
Because of this the chance of malicious attacks are significantly reduced
Because of this the chance of malicious attacks are significantly reduced
💠Meta Chain Shard
The Meta Chain runs seperately & notes all history data for nodes-joining, leaving, invalid blocks, etc
It communicates with all other shards & allows requests info about blocks, etc to securely process cross-shard txs
The Meta Chain runs seperately & notes all history data for nodes-joining, leaving, invalid blocks, etc
It communicates with all other shards & allows requests info about blocks, etc to securely process cross-shard txs
💠Cross-Shard Execution
A wallet sends money from Shard 0 to an address in shard 1 ("Miniblock1")
This is validated in Shard 0 & recorded on the Metachain
It is then seen by shard 1, Miniblock1 is requested, then tx requested, then its validated in Shard 1& recorded on Metachain
A wallet sends money from Shard 0 to an address in shard 1 ("Miniblock1")
This is validated in Shard 0 & recorded on the Metachain
It is then seen by shard 1, Miniblock1 is requested, then tx requested, then its validated in Shard 1& recorded on Metachain
💠Static State Sharding
This partitions blockchain data into fixed sets using predetermined criteria
Each shard has access to the same static state information for parallel transaction processing
Improving scalability and throughput
$ETH 2.0 plans to use this approach
This partitions blockchain data into fixed sets using predetermined criteria
Each shard has access to the same static state information for parallel transaction processing
Improving scalability and throughput
$ETH 2.0 plans to use this approach
💠Static vs Adaptive
>Static= less complex, easier to manage and more predictable. But has poor reaction to sudden surges in transaction volume
>Adaptive= more flexible and can handle sudden spikes in txs. But is more complex & can lead to security issues if not managed well
>Static= less complex, easier to manage and more predictable. But has poor reaction to sudden surges in transaction volume
>Adaptive= more flexible and can handle sudden spikes in txs. But is more complex & can lead to security issues if not managed well
💠Sharding Vulnerabilities
>Single-Shard Takeover Attack- MVX mitigates this using Context Switching
>Cross-shard Communication- Metachain & automated mechanisms used
>Data availability and the need of an abstraction layer that hides the shards- Metachain used
>Single-Shard Takeover Attack- MVX mitigates this using Context Switching
>Cross-shard Communication- Metachain & automated mechanisms used
>Data availability and the need of an abstraction layer that hides the shards- Metachain used
💠MVX one of a kind?
Yes! MVX is an extremely well thought-out L1 Blockchain, that while complicated has some amazing solutions to the Blockchain Trilemma!
Adaptive State Sharding is a big & impressive leap
The tech is ready, the adoption is pending...
Yes! MVX is an extremely well thought-out L1 Blockchain, that while complicated has some amazing solutions to the Blockchain Trilemma!
Adaptive State Sharding is a big & impressive leap
The tech is ready, the adoption is pending...
https://twitter.com/beniaminmincu/status/1528462865551790081?s=20
💠Comparison to Other ⛓️
>ETH 2.0- Mentioned
>Polkadot- Uses Heterogenous Sharding= each shard has their own consensus
>Zilliqa- Uses Network Sharding
>NEAR- Uses State atm but will also do processing in 2023 (validating txs no longer just full nodes)
>ETH 2.0- Mentioned
>Polkadot- Uses Heterogenous Sharding= each shard has their own consensus
>Zilliqa- Uses Network Sharding
>NEAR- Uses State atm but will also do processing in 2023 (validating txs no longer just full nodes)
💠Great Thread:
>
>
https://twitter.com/DBCrypt0/status/1590710825618399233?s=20(A must read & very digestible, unlike mine🤣)
💠Great Resources:
>docs.multiversx.com/technology/ada…
>files.multiversx.com/multiversx-whi…
>32prosperity.medium.com/is-elrond-netw…
#Blockchain #BlockchainTrilemma
>docs.multiversx.com/technology/ada…
>files.multiversx.com/multiversx-whi…
>32prosperity.medium.com/is-elrond-netw…
#Blockchain #BlockchainTrilemma
💠Feedback
If you made it to here, that is impressive! 👏 I owe you a cookie!
(disclaimer: no cookies will be provided under ANY circumstances!)
Please feel free to comment any corrections or points to add 😁
If you enjoyed, Comment, Like & Retweet 🙏
If you made it to here, that is impressive! 👏 I owe you a cookie!
(disclaimer: no cookies will be provided under ANY circumstances!)
Please feel free to comment any corrections or points to add 😁
If you enjoyed, Comment, Like & Retweet 🙏
https://twitter.com/TomosRees2/status/1633841634285883394?s=20
• • •
Missing some Tweet in this thread? You can try to
force a refresh
