Sebastian Aaltonen Profile picture
Oct 5, 2024 30 tweets 6 min read Read on X
Let's talk about CPU scaling in games. On recent AMD CPUs, most games run better on single CCD 8 core model. 16 cores doesn't improve performance. Also Zen 5 was only 3% faster in games, while 17% faster in Linux server. Why? What can we do to make games scale?

Thread...
History lesson: Xbox One / PS4 shipped with AMD Jaguar CPUs. There was two 4 core clusters with their own LLCs. Communication between these clusters was through main memory. You wanted to minimize data sharing between these clusters to minimize the memory overhead.
6 cores were available to games. 2 taken by OS in the second cluster. So game had 4+2 cores. Many games used the 4 core cluster to run your thread pool with work stealing job system. Second cluster cores did independent tasks such as audio mixing and background data streaming.
Workstation and server apps usually spawn independent process per core. There's no data sharing. This is why they scale very well to workloads that require more than 8 cores. More than one CCD. We have to design games similarly today. Code must adapt to CPU architectures.
On a two CCD system, you want to have two thread pools locked on these cores, and you want to push tasks to these thread pools in a way that minimizes the data sharing across the thread pools. This requires designing your data model and communication in a certain way.
Let's say you use a modern physics library like Jolt Physics. It uses a thread pool (or integrates to yours). You could create Jolt thread pool on the second CCD. All physics collisions, etc are done in threads which share a big LLC with each other.
Once per frame you get a list of changed objects from the physics engine. You copy transforms of changed physics engine objects to your core objects, which live in the first CCD. It's a tiny subset of all the physics data. The physics world itself will never be accessed by CCD0.
Same can be done for rendering. Rendering objects/components should be fully separated from the main game objects. This way you can start simulating the next frame while rendering tasks are still running. Important for avoiding bubbles in your CPU/GPU execution.
Many engines already separate rendering data structures fully from the main data structures. But they make a crucial mistake. They push render jobs in the same global job queue with other jobs, so they will all be distributed to all CCDs with no proper data separation.
Instead, the graphics tasks should be all scheduled to a thread pool that's core locked to a single CCD. If graphics is your heaviest CPU hog, then you could allocate physics and game logic tasks to the thread pool in the other CCD. Whatever suits your workload.
Rendering world data separation is implemented by many engines already. It practically means that you track which objects have been visually modified and bump allocate the changed data to a linear ring buffer which is read by the render update tasks when next frame render starts.
This kind of design where you fully separate your big systems has many advantages: It allows refactoring each of them separately, which makes refactoring much easier to do in big code bases in big companies. Each of these big systems also can have unique optimal data models.
In a two thread pool system, you could allocate independent background tasks such as audio mixing and background streaming to either thread pool to load balance between them. We could also do more fine grained splitting of systems, by investigating their data access patterns.
Next topic: Game devs historically were drooling about new SIMD instructions. 3dNow! Quake sold AMD CPUs. VMX-128 was super important for Xbox 360 and Cell SPUs for PS3. Intel made mistakes with AVX-512. AVX-512 was initially too scattered and Intel's E-cores didn't support it. Image
Game devs were used to writing SIMD code either by using a vec4 library or hand written intrinsics. vec4 already failed with 8-wide AVX2, and hand written instrinsics failed with various AVX-512 instruction sets and various CPU support. How do we solve this problem today?
Unreal Engine's new Chaos Physics was written with Intel's ISPC SPMD compiler. ISPC allows writing SMPD code similar to GPU compute shaders on CPU side. It supports compiling the same code to SSE4, ARM NEON, AVX, AVX2 and AVX-512. Thus it solves the instruction set fragmentation.
Unity's new Burst C# compiler aims to do the same for Unity. Burst C# is a C99-style C# subset. The compiler leans heavily on autovectorization. Burst C# has implicit knowledge of data aliasing allowing it to autovectorize better than standard compiler. Same is true for Rust.
However autovectorization is always fragile no matter how many "restrict" keywords you put either manually or by the compiler. ISPCs programming model is better suited for reliable near optimal AVX-512 generation.
ISPC compiles C/C++ compatible object files. They are easy to call from your game engine code. Workloads such as culling, physics simulation, particle simulation, sorting, etc can be done using ISPC to get AVX2 (8 wide) and AVX-512 (16 wide) performance benefits.
The last topic: SMT and Zen 5 dual decoders. Zen 5 has independent decoders for both threads. This helps server workloads. Zen 5 also has wider execution units to sustain running two SMT threads better. Can we design our game code to work better with SMT (Hyperthreading)?
The biggest problem with SMT is actually the same problem that we have with E-cores: Thread time variance. If we assume 130% SMT throughput (vs one thread on CPU), both SMT threads run at 65% performance. So they take 54% longer to finish...
Many game engines still have a main thread and some have a graphics thread too. These dedicated threads often become performance bottlenecks. If any of these critical threads gets scheduled to E-core or some other thread runs on the same core with SMT, we have a problem.
I have a solution: Don't have a main thread at all. Just use tasks that spawn tasks. This way programmers can't write code in main thread. Problem solved? Yes, if you are writing a new engine from scratch. Very hard to refactor original engine to "no main thread" model.
The other problem is that simple schedulers implement parallel for loops by evenly splitting the job to N workers. What if one of these workers is E-core or SMT thread? Other threads finish sooner, but next task has to wait for the slowest E-core/SMT thread to finish...
The solution for this is to use work stealing. For parallel for loops, I recommend using "lazy binary splitting". This balances very well with minimal scheduling overhead. Basically you always steal half of the work instead of fixed amount of work.

dl.acm.org/doi/10.1145/16…
Conclusions: Solutions exist for minimizing multi-CCD data sharing, improving scheduling for E-cores/SMT and cross platform SPMD SIMD programming (AVX2/AVX-512). We need to improve our engine tech to make it more suitable for modern processors. CPUs have changed. Tech must too.
@hkultala And this is for 8 core models in gaming. Games still don't scale to 16 cores properly as it requires game engine changes. Same with E-cores and SMT. Performance benefit could be bigger if engine architecture was modified.
@AgileJebrim Also static load balancing on modern cache hierarchies is difficult. So many different cache levels. 200 cycle memory latency on a 6 wide system means up to 1200 instruction stall for a cache miss. This is dynamic behavior. You can't predict cache misses statically.
@AgileJebrim You want to threat a 16 core AMD Zen CPU similarly as a dual GPU setup. You don't want to split parallel for between them, as then results are 50/50 split in their memories. Next step needs to do mixed reads from two memories, if access pattern doesn't match exactly.
@AgileJebrim If you want to statically allocate the workload so that it fits both of these CPUs, you have to limit your CPU workload to two medium performance cores. You lose performance on both the dual-core iPhone 6 and 7 and you also lose all E-cores on the Android...

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Sebastian Aaltonen

Sebastian Aaltonen Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @SebAaltonen

May 30
The past decades have been a wonderful time for gamers+devs. The biggest chips, using the latest nodes and trillions worth of R&D, were all targeted at gaming. Now, those chips are needed by professionals (AI). We'll never see a big die GPU at a reasonable price point anymore :(
The fun lasted for a very long time, but it's over in both CPU and GPU side. The biggest CPU and GPU dies are no longer designed for gamers. Top end Threadripper costs over 10k$ today. Top end Nvidia B200 costs over 30k$. Few generations ago top tier HW was targeting gamers :(
AMD no longer produces big-die GPUs for gamers. Nvidia has a low-volume 2500$+ Halo product. But it's much smaller than Nvidia's B200 GPU, which has two glued dies, each slightly bigger than RTX 5090. Chiplet GPUs like Threadripper are coming. Gaming GPUs limited to few chiplets?
Read 4 tweets
May 18
Unit tests have lots of advantages, but cons are ignored:
- Code must be split to testable parts. Often requiring more interfaces, which add code bloat and complexity.
- Each call site is a dependency. Test case = +1 dependency. Added inertia to refactor and throw away code.
...
- Bloated unit test suites taking several hours to execute. Slows down devs and causes merge conflicts as pushes are delayed.
- Unstable tests randomly failing pushes.
- Unit test maintenance and optimization needed to keep tests manageable. Otherwise developer velocity hurts.
It's crucial to make your unit tests fast. Don't load files from disk and definitely don't do network requests. Embed data (bin->hdr tool for example). If your whole test suite runs in <10 seconds, then you are golden. But writing good optimized tests like this takes effort.
Read 9 tweets
May 7
When you split a function to N different small functions, the reader also suffers multiple "instruction cache" misses (similar to CPU when executing it). They need to jump around the code base to continue reading. Big linear functions are fine. Code should read like a book.
Messy big functions with lots of indentation (loops, branches) should be avoided. Extracting is a good practice here. But often functions like this are a code smell. Why do you need those branches? Why is the function doing too many unrelated things? Maybe too generic? Refactor?
There's a rule of thumb that you write separate code for each call site until you have repeated yourself 3 times. Then you merge these together. But people often forget the opposite: You have to split a function if the call site requirements change. Don't add more branches!
Read 5 tweets
Jan 23
WebGPU CPU->GPU update paths are designed to be super hard to use. Map is async and you should not wait for it. Thus you can't map->write->render in the same frame.

wgpuQueueWriteBuffer runs in CPU timeline. You need to wait for callback to know buffer is not in use.

Thread...
Waiting for callback is not recommended in web and there's no API for asking how many frames you have in flight. So you have to dynamically create new staging buffers (in a ring) based on callbacks to use wgpuQueueWriteBuffer safely. Otherwise it will trash data used by GPU.
You are not allowed to map or wgpuQueueWriteBuffer a different region of a buffer used by any GPU frame in flight. You need entirely different buffer.
Read 10 tweets
Dec 17, 2024
I would really love to write a blog post about this whole debate. This is a too complex topic to discuss on Twitter.

Ubisoft was among the first to develop GPU-driven rendering and temporal upscaling. AC: Unity, Rainbow Six Siege, For Honor, etc. Our SIGGRAPH 2015 talk, etc...
Originally, TAA was seen as an improvement over screen space post-process AA techniques as it provided subpixel information. It wasn't just a fancy blur algo. Today, people render so much noise. Noise increases neighborhood bbox/variance. Which increases ghosting.
Today people don't do 1:1 TAA anymore. TAA shader does upscaling too. Thus you have maybe 1:4 samples compared to before. And much noisier signal. Plus people feed this reconstructed image to frame interpolator, which hallucinates even more data. This all combined is the problem.
Read 6 tweets
Nov 22, 2024
AMD UDNA will be interesting.

CDNA3 architecture is still based on GCN 4 cycle wave64 scheduling. RDNA schedules every cycle and exposes instruction latency. Scheduler runs/blocks instructions concurrently/dynamically. RDNA is much closer to Nvidia GPUs.

Thread... Image
CDNA has wide matrix cores and other wide compute workload improvements, which AMD wants to bring to UDNA. It also has multi-chip scaling.

Rumors tell that RDNA4 will finally have matrix cores in consumer space. Seems that AMD is integrating matrix cores early to RDNA lineup.
My expectation is that UDNA compute unit will be RDNA4 descendant instead of CDNA3 descendant. They definitely need 1 cycle low latency scheduling in consumer space, and Nvidia does well with it in AI space too. I don't see them going back to GCN-style design for UDNA.
Read 8 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(