Thread by @jcksie

The medalla #eth2 testnet hasn't been finalizing for a while which puts some additional stress on clients as the number of possible valid chain histories, or branches, grows - here's what goes on in an @ethnimbus client in this case:

* We use more memory - normally we hover around 500mb but memory consumption now goes up to 1-2gb: In each possible history, votes and validator assignments might be different - we keep a compressed in-memory cache of the most crucial information

The caches are used to process attestations - the network becomes unhealthy if attestation processing is slow or if we have to drop some because we don't have time to recreate the committee to validate them - for this we need access to the shuffling of active validators

Another source of memory usage is the number of branches itself - depending on when the client was started, it will have seen more of less of these - each potential branch takes up some space, and we currently keep all of them around

To manage the branch memory usage, we are implementing pruning of unviable branches - if at some point a branch was created but nobody is building on it, it's unlikely to become the canonical history and we can limit memory waste by dropping it on a probabilistic basis

* We use more disk space - per the protocol, we must be prepared for clients voting for any of the branches and this number keeps growing - to avoid keeping everything in memory, we store them on disk

Storing non-finalized states on disk allows us to recreate any state in constant time, regardless of how long the non-finality is ongoing, and thus decreasing the negative performance impact on the network as a whole

The disk space will eventually be reclaimed as we prune the database, but for that to happen, the chain needs to finalize - after finalization, any alternate history becomes invalid and we can drop it

A few finalized state snapshots are kept for RPC requests - we might have to replay more blocks to recreate a particular finalized state but this will never be needed for the eth2 network and protocol itself, thus we prefer to save on disk space

* Sync becomes slower: when we connect to other peers, they are more likely to not be following the same head - this in turn makes it harder to know that we are synced, so we will spend more time exploring the different histories in the network

Sometimes, we might also have to resync parts, in case the peers we're connected to disagree on which history is valid - in benign cases they might be slow or have poor connectivity with parts of the network - in malicious cases, they might be trying to eclipse us

All in all, the long period of non-finality has served to tune and optimize the client to better handle this unlikely, but potentially disabling scenario - the improvements not only make the client more efficient in normal scenarios but also increase the security buffer ..

.. we have against future attacks on the network as a whole!