Thread by @0xb17z

For those of you who still have @Delphi_Digital's Hitchhiker's Guide to Ethereum open somewhere in your list of 100 tabs in your browser, this thread is for you! I try to distill it down from 1 hour to about 15 minutes. However, y'all should absolutely read it. 🧵👇

1/58

@Delphi_Digital Yes, it's going to be long! But first thing's first, you should have this open and a thank you to @jon_charb for putting this together!

members.delphidigital.io/reports/the-hi…

2/58

@Delphi_Digital @jon_charb Now, let's establish some background information.

@VitalikButerin's Limit to Scalability Post: - vitalik.ca/general/2021/0…
This is to provide context on what his standards are for a chain to be sufficiently trustless and censorship resistant.

3/58

@Delphi_Digital @jon_charb @VitalikButerin Also, his Endgame Post: vitalik.ca/general/2021/1…

This is to establish an understanding on what he thinks is a plausible endgame for Ethereum in regards to his standards.

4/58

@Delphi_Digital @jon_charb @VitalikButerin Also, his post on Why Sharding is Great: vitalik.ca/general/2021/0…

This is to provide some context on scaling the layer 1 aspect of Ethereum

5/58

@Delphi_Digital @jon_charb @VitalikButerin And to recall the rollup-centric roadmap for context on layer 2 scaling: ethereum-magicians.org/t/a-rollup-cen…

6/58

@Delphi_Digital @jon_charb @VitalikButerin Now, for the themes of this thread:

• "Sufficient Decentralization & Vitalik's Endgame" will go further into his ideas of what that theoretical chain looks like

• "Sharding: The Old Way and the Dank New Way" will go into danksharding and data availability sampling

7/58

@Delphi_Digital @jon_charb @VitalikButerin • "Rolling on Up" will go into erasure coding as a solution for data availability sampling as well as proto-danksharding

• "Data, Data, and More Data. Please, Enough Data" will go into the types of data in Ethereum and ways to deal with them

8/58

@Delphi_Digital @jon_charb @VitalikButerin • "MEV" will go into designing a protocol with MEV in mind.

Finally, I want to point out once again, you should read the Hitchhiker's Guide and that if I am unclear or misunderstanding something, please, let me know. I'm open to feedback and want to get better.

9/58

@Delphi_Digital @jon_charb @VitalikButerin Last thing: I will not go over the math in guide. I think they do a good job of it so I will just bring up analogies (but be careful about becoming too attached to analogies). Alright, finally, let's get started!

10/58

@Delphi_Digital @jon_charb @VitalikButerin Sufficient Decentralization and Vitalik's Proposed Endgame:

Vitalik's standards on decentralization is that the users are the decentralization. So, it is crucial for anyone to run a node and that means having a culture of users validating the blockchain.

11/58

@Delphi_Digital @jon_charb @VitalikButerin In his Endgame Blog Post, he outlines 4 points on what it would take to make a chain sufficiently trustless and censorship resistance:

1. Adding a second tier of staking that makes it easy for anyone to do distributed blockchain validation.

12/58

@Delphi_Digital @jon_charb @VitalikButerin 2. Have fraud proofs or ZK-SNARKs to let users check block validity

3. Data availability sampling (DAS) to let users check block availability.

Now, let's do a sidebar on availability: We need to know data is complete and correct.

13/58

@Delphi_Digital @jon_charb @VitalikButerin However, there's a type of attack called a data withholding attack that pretty much guarantees nodes needing to have to store all this data to prevent it. But didn't we just say, "It's crucial for anyone to run a node?" Well, that's a current blocker to scaling.

14/58

@Delphi_Digital @jon_charb @VitalikButerin The goal of DAS is to reduce how much data needs to be downloaded via a sampling.

Back to 4th point:
4. Add secondary transaction channels to prevent censorship

15/58

@Delphi_Digital @jon_charb @VitalikButerin Thus, he concludes all roads end with partial centralization: The 4 above bullets lead to a chain where block production is centralized, but block validation is trustless and there is a mechanism in place that prevents block producers from censoring.

End of section

16/58

@Delphi_Digital @jon_charb @VitalikButerin Sharding: The Old Way and The Dank New Way:

Let's recall some themes from the previous section:
• Data availability
• Data availability sampling
• The users are the decentralization
• Censorshop resistancce

17/58

@Delphi_Digital @jon_charb @VitalikButerin The old way of sharding proposed partitioning the ETH2 blockchain into 64 shards. Each of the shards had separate proposers and committees that were on rotation from the validator set and each individually have to verify that their shard's data was made available.

18/58

@Delphi_Digital @jon_charb @VitalikButerin The problem with this model is that voting can't always be completed in a single slot and this is quite complex of an architecture.

19/58

@Delphi_Digital @jon_charb @VitalikButerin In danksharding, validators conduct data availability sampling to confirm all data is available. So, instead of there being a fixed number of a shards that have distinct block proposers, there's this merged-fee market which just has one proposer

20/58

@Delphi_Digital @jon_charb @VitalikButerin This proposer is a specialized builder that creates one large block with all the transactions and data that go in a slot.

The best part? Everything is done on the main chain which greatly simplifies the architecture.

21/58

@Delphi_Digital @jon_charb @VitalikButerin HOWEVER, Remember the theme of users are decentralization? Building that block is resource heavy so there's this proposal of a Proposer Builder Separation to keep danksharding decentralized

22/58

@Delphi_Digital @jon_charb @VitalikButerin In that model, we have:

• Block builders bidding on the right to choose contents of a slot (this is a new role built into in the protocol)
• Proposers (validators) that select the valid header with the highest bid

and

23/58

@Delphi_Digital @jon_charb @VitalikButerin • All other validators and users that can verify the blocks through.... data availability sampling

24/58

@Delphi_Digital @jon_charb @VitalikButerin Builders will put together blocks and bid for proposers to select their block, which should block MEV centralization (more in this later). Competitive builders will extract as much as they can and the value will, in theory, trickle down to the decentralized validator set.

25/58

@Delphi_Digital @jon_charb @VitalikButerin he exact implementation of how this is going to work is still being discussed. For a proposed solution, check out: ethresear.ch/t/two-slot-pro…

26/58

@Delphi_Digital @jon_charb @VitalikButerin However, PBS gives builders an ability to censor transactions. To combat that, we have Censorship Resistance Lists where proposers specify a list of all eligible transactions and the builder will be forced to include it. Implementation is also TBD.

End of Section

27/58

@Delphi_Digital @jon_charb @VitalikButerin Rolling On Up:

Recall the themes:
• Data availability
• Data availability sampling
• Users are the decentralization and it should be simple to run a node

28/58

@Delphi_Digital @jon_charb @VitalikButerin Rollups post so much data... but remember that it's a blocker to scaling if nodes have to download all of it. So data availability sampling allows nodes to easily and securely verify that all of the data is available without having to download it.

29/58

@Delphi_Digital @jon_charb @VitalikButerin However, the solution is not firing blind and checking a bunch of random data chunks on the block. Important data might be missed. So, the way forward for this is using erasure coding (this is where @jon_charb gets into math in the guide - check it out).

30/58

@Delphi_Digital @jon_charb @VitalikButerin TL;DR on erasure coding: You encode data blobs (pieces of data) that if at least half the data in a blob if published, anyone in the network can reconstruct the data.

31/58

@Delphi_Digital @jon_charb @VitalikButerin Think of it like a recipe to cook something. If you are presented with half the ingredients, you're likely to be able reconstruct the whole recipe.

32/58

@Delphi_Digital @jon_charb @VitalikButerin In math terms, if you have line with 4 points, if you have 2 of those points presented, you can reconstruct the line and find the 2 remaining points.

33/58

@Delphi_Digital @jon_charb @VitalikButerin If you want to go deeper, you can look into Kate Committments, which is a version of Polynomial Commitments: dankradfeist.de/ethereum/2020/…

34/58

@Delphi_Digital @jon_charb @VitalikButerin TL;DR: Merkle Roots are used to commit to large amounts of data but this is only useful if proving that some data exists in a full data set. You also need to know about all the other data and if they lie "in the same plane" (think about the line example I mentioned above)

35/58

@Delphi_Digital @jon_charb @VitalikButerin PBS and Danksharding go hand in hand because regular validators can't handle the massive bandwidth of a block full of rollup data.

36/58

@Delphi_Digital @jon_charb @VitalikButerin A specification for moving towards dansksharding - Proto-danksharding (EIP-4844) eips.ethereum.org/EIPS/eip-4844

Rollups are considered the only trustless scaling solution for Ethereum. Rollups also significantly reduce fees

37/58

@Delphi_Digital @jon_charb @VitalikButerin • Optimism & Arbitrum usually are 3-8x lower
• ZK rollups usually are 40-100x lower because they have better data compression

38/58

@Delphi_Digital @jon_charb @VitalikButerin However these fees are still considered too expensive to users. The long term solution is data sharding which would add around 16 MB per block of dedicated data space to the chain that rollups can use

39/58

@Delphi_Digital @jon_charb @VitalikButerin The spec includes:
• All of the execution layer logic required for danksharding
• Layer separation between BeaconBlock verification and data availability sampling blogs
• Beacon Block logic for danksharding
• Much more, check it out

40/58

@Delphi_Digital @jon_charb @VitalikButerin Today rollups use calldata (all the data passed to smart contract at execution time). In the future, rollups will have to use sharded data (blobs or pieces of data) because it's much cheaper and guarantees data availability.

End of section

41/58

@Delphi_Digital @jon_charb @VitalikButerin Data, Data, and More Data. Please, Enough Data:

We have historical data, which is basically everything that's ever happened on-chain.

42/58

@Delphi_Digital @jon_charb @VitalikButerin We also have state data which is a snap shot of:

• Current account balances
• Account nonces
• Smart contract code
• Smart contract storage
• Consensus-related data

43/58

@Delphi_Digital @jon_charb @VitalikButerin These can get really large so putting the burden on the nodes makes them much harder to run. And remember the theme? Users are decentralization so we need ways of mitigating this.

44/58

@Delphi_Digital @jon_charb @VitalikButerin Enter: EIP-4444 eips.ethereum.org/EIPS/eip-4444
• Prune historical data in clients older than one year
• Note: Proto danksharding prunes blobs after abouth a month

45/58

@Delphi_Digital @jon_charb @VitalikButerin It is not the job of the Ethereum protocol to maintain all this historical data (only state data) since this only proposes a risk to the applications on top of the protocol. So now, where will this data go? Other protocols, block explorers, data services, etc.

46/58

@Delphi_Digital @jon_charb @VitalikButerin The primary bottleneckto Ethereum's TPS is the state data. So, Ethereum is trying to go for "weak statelessness" meaning state isn't required to validate a block, but it is required to build the block.

47/58

@Delphi_Digital @jon_charb @VitalikButerin Weak statelessness abstracts the state data constraints away from validators.

The following example scenario is taken straight from the guide:

48/58

@Delphi_Digital @jon_charb @VitalikButerin If I want to send a transaction to Alice, we need to know:

• I have 1 ETH before the transaction
• My public key
• My nonce (transaction ordering)
• I have 1 fewer ETH and Alice has 1 more ETH after the transaction

49/58

@Delphi_Digital @jon_charb @VitalikButerin The builder adds this "witness data" to the block and the proof of its accuracy and the validator just receives the block, executes it, and decides if it's valid.

End of section

50/58

@Delphi_Digital @jon_charb @VitalikButerin MEV:

• Recall PBS
• Reminder: MEV (maximal extractable value) refers to the max value that can be extracted from block production in addition to the standard block reward and gas fees by including, excluding, and changing the order of transactions in a block

51/58

@Delphi_Digital @jon_charb @VitalikButerin Proposer-Builder Separation was actually first designed to combat the centralizing forces of MEV but will not be ready at the merge

52/58

@Delphi_Digital @jon_charb @VitalikButerin Designing a protocol with MEV in mind contains two major points:

1. Mitigate Harmful MEV

Example 1: Try to reorg the chain and steal MEV from a previous block it. So, achieve finality in a single slot (far on the roadmap)

53/58

@Delphi_Digital @jon_charb @VitalikButerin Example 2: Beacon Chain Validators lists and upcoming leader selection lists are public. Smaller validators can get doxxed and DDOS'd easier and thus can be exploited for MEV.

54/58

@Delphi_Digital @jon_charb @VitalikButerin Potential solution: Single Secret Leader Election where nobody but the proposer knows when their turn is up (not live at the merge)

55/58

@Delphi_Digital @jon_charb @VitalikButerin 2. Democratize the rest. Remember: users are the decentralization.

56/58

@Delphi_Digital @jon_charb @VitalikButerin Since PBS won't be ready for the merge, there's MEV-Boost

• It will plug into consensus clients allowing outsourcing of block building which is a step on the way to PBS.

End of section

57/58

@Delphi_Digital @jon_charb @VitalikButerin Well, that's it! If you made it this far, do yourself a favor and read @jon_charb's guide. And once again, if there's anything I missed or misunderstood, I'm all eyes and ears. Thank you for taking the time to read and hopefully it was helpful.

58/58