Thread by @SebAaltonen

A modern gaming CPU such as the 5950X has achievable memory bandwidth of 35 GB/s. At 144 fps that's 35 GB/s / 144 fps = 243 MB/frame. That's how much unique memory you can access in 1 frame.

5800X3D has 96 MB of LLC. Almost the whole working set fits to the cache. Thread...

That 243 MB/frame figure is highly optimistic. It assumes that everything your run on the CPU is bandwidth bound, and it assumes you never access the same data twice. It's common to produce the data first and consume later in the frame. You access the same cache lines twice.

So it's likely that we already have games that fit their whole working set in the 96 MB cache of 5800X3D, assuming the game runs at 144 fps of course. Since games are highly temporally coherent, there's only a few percent change in the working set between the frames.

I think this is the reason why 5800X3D is so fast in many games. The LLC is finally large enough to contain the whole working set, especially for games designed to run at 144 fps or higher. All data accessed in previous frame are already in the cache, improving the latency A LOT.

Since 120Hz and 144Hz seem to be the new normal for gaming, it's iteresting to see whether the CPU vendors will continue on this path. Aiming to provide caches large enough to contain the whole working set, being able to provide cache hit for all last frame data.

Some games double buffer their data, some games mutate the current data directly. Mutating one set of data results in smaller working set, but is trickier to parallelize.

Frame temp allocators (bump allocators with frame life time) can also increase the working set, as there's a conservative assumption that all temp data lives until end of the frame, and those memory regions are not reused during the frame.

If CPUs with massive LLCs become common, we might need to reconsider the way we implement our data processing. Minimizing the working set and ensuring the working set is temporally coherent (next frame) becomes crucial for best performance.