Thread by @TimBeiko

Better late than never, another @ethereum #AllCoreDevs recap. Tons of Merge updates on today's call!

Agenda: github.com/ethereum/pm/is…
Stream:

Recap below 👇🏻

https://twitter.com/TimBeiko/status/1520122228355993600

@ethereum First, once more, shoutout to @christine_dkim for having a recap out before me - strongly recommend for another perspective on the call 😁

https://twitter.com/christine_dkim/status/1525178255002677248

@ethereum @christine_dkim We kicked off the call with a recap of the fourth mainnet shadow fork, by @parithosh_j. TL;DR: it went pretty well! We did see a few bugs, some of which weren't merge related. The chain did finalize very smoothly 👌🏻

https://twitter.com/parithosh_j/status/1524767841773363200

@ethereum @christine_dkim @parithosh_j The main issue we saw was empty blocks being produced by some EL clients because some CL clients were asking for them "too quickly".

@ethereum @christine_dkim @parithosh_j To expand on this: the way blocks get created post-merge is that a validator who is chosen to produce the next block has their CL sends a first API call to the EL to let them know about the latest head, and have them start building a candidate block on top of that head.

@ethereum @christine_dkim @parithosh_j The CL must then another call to the EL for it to return its candidate block. If the first and second call are two close to one another, the EL returns an empty block to the CL because it didn't have time to create one from the transactions in its txn pool.

@ethereum @christine_dkim @parithosh_j Affected CL teams are working on a fix for this (basically delaying the time between the two calls), and @prylabs already had it out by the ACD call.

@ethereum @christine_dkim @parithosh_j @prylabs One thing worth noting is that while this obviously isn't ideal behaviour, the chain still progresses as the EL returns a block with 0 transactions, rather than an empty block, like we sometimes see on PoW.

@ethereum @christine_dkim @parithosh_j @prylabs Additionally, given EIP-1559's slack mechanism, if these blocks don't happen twice in a row, the throughput of the chain stays constant (although inclusion time worsens for txns who could have been in the empty block).

@ethereum @christine_dkim @parithosh_j @prylabs Not to say this isn't a bug and that it won't be fixed 😅 Just providing some context.

@ethereum @christine_dkim @parithosh_j @prylabs Aside from this, a sync issue on Erigon nodes, which were using a new sync mechanism (more on that here: erigon.substack.com/p/erigon-2-thr…), prevented them from catching up to the tip of the chain in time for the fork. It's worth noting that Erigon's previous sync mode still works as usual

@ethereum @christine_dkim @parithosh_j @prylabs All in all, this was a very smooth shadow fork. The next one, planned for next week, will have a different configuration: instead of using client shares comparable to mainnet (see:

https://twitter.com/TimBeiko/status/1524136595862941696

), we'll have an equal split across EL & CL clients for validators.

@ethereum @christine_dkim @parithosh_j @prylabs Next up on the call, we discussed a clarification to the Engine API for how an EL should respond if it receives an invalid candidate merge block from the CL. Usually, when the EL receives an invalid block, it tells the CL what the last valid parent block it has seen is.

@ethereum @christine_dkim @parithosh_j @prylabs In this case, though, because the parent would be a PoW block, the CL won't have it in its block tree, and so it's useless to return a specific block hash. We simply agreed to return 0x00..00 as the hash instead. More details here: github.com/ethereum/execu…

@ethereum @christine_dkim @parithosh_j @prylabs After this, we also discussed when we should expect the Engine API, as well as other specs, to be frozen. In short, basically now 🥶

@ethereum @christine_dkim @parithosh_j @prylabs We don't expect to make more substantive changes to either the Engine API, EL or CL specs. There are a few clarifications we want to add to them, but expect to have release candidates cut for all of them next week.

@ethereum @christine_dkim @parithosh_j @prylabs After this, we covered additions to JSON RPC block tags. In short, at The Merge, a `finalized` tag will be added, which allows users to get the last finalized block on the chain (or an error if there isn't one, meaning The Merge hasn't happened yet).

@ethereum @christine_dkim @parithosh_j @prylabs Marius already has it implemented in Geth 👀

https://twitter.com/vdWijden/status/1525067776796262400

@ethereum @christine_dkim @parithosh_j @prylabs Along with this, we'll also introduce a `safe` tag. For simplicity, now it will likely alias to `finalized` across client implementations. That said, we can eventually upgrade it to return a block which we expect to not revert under normal network conditions, usually ~4s old.

@ethereum @christine_dkim @parithosh_j @prylabs The `latest` tag will remain, and keep returning the most recent block the chain has seen, regardless of whether it can reorg. The changes to the spec are available here: github.com/ethereum/execu…

@ethereum @christine_dkim @parithosh_j @prylabs After this, we had a long conversation about the pros/cons of setting a specific Fork ID on the EL for The Merge. Recommend the livestream for all the back and forth, but I'll try and summarize here!

@ethereum @christine_dkim @parithosh_j @prylabs First, Fork ID, introduced in EIP-2124 is a way for peers on a same network (i.e. with the same *Chain* ID) to disconnect from each other if they have a different fork history eips.ethereum.org/EIPS/eip-2124

@ethereum @christine_dkim @parithosh_j @prylabs Fork ID is taken by creating a checksum from the genesis hash of the network and all the fork blocks which the node has already passed. Nodes also store a value if they are aware of a future fork happening for them, called FORK_NEXT.

@ethereum @christine_dkim @parithosh_j @prylabs This way, if you see a peer who has a FORK_NEXT for a block you've already passed locally and haven't forked on, you can know to disconnect them as they are or will be on another chain.

@ethereum @christine_dkim @parithosh_j @prylabs We always use this during network upgrades, but the challenge with this one is that we are *not* using a block height to trigger The Merge, but a Total Difficulty value, which isn't compatible with this scheme.

@ethereum @christine_dkim @parithosh_j @prylabs So, we have a few options:
1. Re-arrange the Fork ID scheme to support difficulty
2. Use a FORK_NEXT block which we estimate happens _before_ the merge and disconnect peers before TTD
3. Use a FORK_NEXT block which we estimate happens _after_ the merge, and disconnect after TTD

@ethereum @christine_dkim @parithosh_j @prylabs (4) Do nothing with FORK_NEXT for The Merge

@ethereum @christine_dkim @parithosh_j @prylabs (1) was decided against - it's error prone, may break tooling which relies on this, and would only be useful this time around.

@ethereum @christine_dkim @parithosh_j @prylabs (2) is hard to get right because there's a fairly short window between releasing clients and hitting the merge. We could end up with (3) accidentally! Even if that isn't the case, it's also bad because we may disconnect peers who have not upgraded _yet_ but will do so.

@ethereum @christine_dkim @parithosh_j @prylabs Even more so, miners may choose to not upgrade their software for The Merge (why would they?), so it could lead to nodes disconnecting from miners, before the merge... not great.

@ethereum @christine_dkim @parithosh_j @prylabs (3) could work, but if you try and time it for "soon" after the merge, then you may have the same problems as with (2): you get it wrong, it happens before TTD, and you disconnect the miners then. Bad!

@ethereum @christine_dkim @parithosh_j @prylabs And then, if you put (3) very far out, it means that it's basically useless, as until you hit FORK_NEXT, you're still connected to bad peers.

@ethereum @christine_dkim @parithosh_j @prylabs We were spending a lot of time on this, and decided that for Ropsten specifically, it would be simpler to just do (4) - nothing.

@ethereum @christine_dkim @parithosh_j @prylabs We'll go with that and see if the number of bad peers is manageable. If it is on Ropsten, where nodes have little incentive to update, it should also be on mainnet, where infrastructure providers, validators and users have a strong incentive to upgrade their nodes.

@ethereum @christine_dkim @parithosh_j @prylabs If we find any issues on Ropsten with this, then we can spend more time trying to estimate a good value for a FORK_NEXT soon after hitting TTD (option 3), and test it on other testnets before going to mainnet.

@ethereum @christine_dkim @parithosh_j @prylabs Next up, we discussed if and when to fork Ropsten! This was debated for a while so, again, I strongly recommend watching the livestream.

@ethereum @christine_dkim @parithosh_j @prylabs The gist of it is that while clients have code that works, there are still edge cases and bugs and, consequently, what is ready today is *not* what would go on mainnet.

@ethereum @christine_dkim @parithosh_j @prylabs The question is then: do we want to wait to deploy on Ropsten, or is it better to do it sooner and have the broader ecosystem run through the upgrade and get ready for releases across other testnets (and eventually mainnet!).

@ethereum @christine_dkim @parithosh_j @prylabs The Merge is a bit more complex for stakers/infra providers/node operators than other upgrades because they need to do more than just upgrade one piece of software: they need to add a CL to their EL or vice-versa, and make sure their setup still works as expected.

@ethereum @christine_dkim @parithosh_j @prylabs It's also worth noting that Ropsten isn't expected to be maintained post-merge, and is generally pretty unstable, so it is the testnet with the lowest stakes to upgrade.

@ethereum @christine_dkim @parithosh_j @prylabs After some back and forth, we decided to go ahead with it! This will be a good chance for the community to experience the transition, and learnings from this can be incorporated into the next testnet merges (Goerli & Sepolia), which would happen when we have "final" client code.

@ethereum @christine_dkim @parithosh_j @prylabs We expect to have client releases out in about two weeks, along with information about what stakers, node operators, tooling/infra providers need to do for the upgrade. Ropsten is expected to merge around June 8th!

@ethereum @christine_dkim @parithosh_j @prylabs If you do want to get ready before then, I'll once more recommend trying Kiln out: blog.ethereum.org/2022/03/14/kil…

The Merge has already happened on it, but the general node architecture is the same as it will be on Ropsten, other testnets, and mainnet 😁

@ethereum @christine_dkim @parithosh_j @prylabs A quick note about the difficulty bomb 💣 If, after the Ropsten fork, we don't feel ready to move to other testnets, we can still push back the bomb on mainnet at that point. The quality bar for Goerli/Sepolia will be higher than for Ropsten which, again, is being deprecated.

@ethereum @christine_dkim @parithosh_j @prylabs One thing astute readers may notice is that Ropsten (and Sepolia) currently *don't* have beacon chains associated with them! We also discussed how to approach launching these (before the Ropsten TTD stuff, actually, my notes are out of order 😅).

@ethereum @christine_dkim @parithosh_j @prylabs First things first, we decided to just name them the Ropsten and Sepolia beacon chains, with an eye to aliasing Prater to Goerli eventually.

Bobsten, Bepolia and Boerli unfortunately did not make it 😂

@ethereum @christine_dkim @parithosh_j @prylabs There are a few things we wanted to consider when launching these beacon chains: how many validators to have, who runs them, when do they run through Genesis/Altair/Bellatrix. Again, livestream has the nuance, but I'll summarize the outcomes.

@ethereum @christine_dkim @parithosh_j @prylabs For Ropsten, we'll have a set of 100k validators controlled by client & testing teams, but also provide a faucet for community members who want to run a validator to have the option to do so. This way, they can run through The Merge in a relatively stable PoS context.

@ethereum @christine_dkim @parithosh_j @prylabs For Sepolia, we'll go with a much smaller validator set which is controlled mostly by client/infra/testing teams, a bit like Clique networks today. The idea is that these can provide a more stable testing environment for application developers.

@ethereum @christine_dkim @parithosh_j @prylabs And, for Goerli, we'll obviously maintain Prater, which has a great mix of community participation, but also a large % of validators run by client teams which ensure general stability for the network.

@ethereum @christine_dkim @parithosh_j @prylabs It's worth noting that for Ropsten at least, we'll be using an ERC20 version of the deposit contract (thanks @protolambda!) which is a bit easier to manage. It doesn't affect testing, either for The Merge or subsequent features such as withdrawals.

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda Finally, it's expected that for Ropsten and Sepolia, we'll have the beacon chains run through Genesis -> Altair -> Bellatrix in quick succession. For Prater, a separate Bellatrix upgrade will need to be scheduled.

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda And that was it for The Merge! TL;DR: Ropsten merges early June, and we launch 2 new beacon chains before then.

If we're happy with the client code quality & testing coverage at that point, we may move forward to other testnets, otherwise, we can still delay the bomb 💣

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda To close the call, we had planned discussion for 3 EIPs. The first was by Felix from Geth: eips.ethereum.org/EIPS/eip-4938

This simply proposes a new networking protocol version which drops GetNodeData, used as a part of fast sync but no longer necessary for snap sync.

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda Supporting this protocol means that we can start to lay the groundwork for deprecating previous eth/xx versions which support them. All clients are supportive of introducing the new version, but most teams shared they need quite some time before they can deprecate eth/66.

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda Next up, we had someone scheduled to talk about github.com/ethereum/EIPs/… but... they didn't show up. Surprisingly, this was my first no-show on an ACD call!

Anyways, the EIP proposes to increase the SSTORE cost for going from a zero to non-zero value to 40k gas.

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda Then, to finish, we had @xinbenlv on the call to talk about a proposal for expiring transactions, EIP-5081: eips.ethereum.org/EIPS/eip-5081

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda @xinbenlv He had a presentation that he shared on the call to gather feedback from attendees: docs.google.com/presentation/d…

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda @xinbenlv The general feedback was that while the DoS risks may not be as big of a concern as expected, there have been several expiring transaction proposals in the past and typically they don't end up as the #1 thing on our priority list, so don't get included in forks.

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda @xinbenlv Additionally, it seems like these could be done either with the introduction of EIP-3074, or even today by using the "account abstraction via the mempool" construction of EIP-4337 eips.ethereum.org/EIPS/eip-4337

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda @xinbenlv And that was it! Managed to end only ~5 minutes overtime 😅

Next call is scheduled for May 27, 14:00 UTC. See you then 👋🏻

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda @xinbenlv Quick correction (h/t @gakonst), the tweet below should read "rather than an empty slots", not blocks. It's a 0 txn block, included after 12s, *not* "no blocks for a 24s period".

https://twitter.com/TimBeiko/status/1525234296457375745?s=20&t=WJEFIf6qj3T0RQFXnAAb4A

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda @xinbenlv @gakonst And then the long sentence loses a ton of clarity...! Better: "the chain progresses as the EL returns a block with 0 txn, like we sometimes see on PoW, rather than skipping a slot"

@ethereum @christine_dkim @parithosh_j @prylabs @protolambda @xinbenlv @gakonst More context by Proto on the ERC 20 deposit contract 👀

https://twitter.com/protolambda/status/1525249993581416448