We've been playing with the GC knobs @maoni0 added in .NET Core. By tweaking how much Gen 0 budget and heap count we want on 48 core machines for smaller apps, we have a lot better memory usage.
No more super lazy semi-unbounded gen 0 growth making things hard to get a read on:
No more super lazy semi-unbounded gen 0 growth making things hard to get a read on:
@maoni0 One downside with some of the tweaks on the various knows is it's not super well documented (somewhat by design) and I had to poke at which limits were global, per-heap, etc. After we get some of this shipped, I'll be submitting some PRs on the .NET documentation to help others.
@maoni0 For context: on a multi-tenant machine (e.g. many apps on a web tier), lazy collection is problematic.
Example: we have a web tier with 64 GB. Some apps are using up to 4GB when they need a few hundred meg. Is it a bug? A leak? Or is GC just being lazy? No clue without dumps.
Example: we have a web tier with 64 GB. Some apps are using up to 4GB when they need a few hundred meg. Is it a bug? A leak? Or is GC just being lazy? No clue without dumps.
@maoni0 Note: collection is globally triggered when memory is around 80-85%, which signals memory pressure to all apps and GC kicks in. But a leak or growth issue may go undetected next to other apps for way too long. We need predictability to monitor and access. Here's that web tier:
@maoni0 The GC knobs are documented here: github.com/dotnet/coreclr…
For environmental variables, you're prefixing with "COMPLUS_".
Note: if you want to see where they are used, you need to manually grep gc.cpp, since it's so large, GitHub doesn't index it: github.com/dotnet/coreclr…
For environmental variables, you're prefixing with "COMPLUS_".
Note: if you want to see where they are used, you need to manually grep gc.cpp, since it's so large, GitHub doesn't index it: github.com/dotnet/coreclr…
@maoni0 For a breakdown of that app - here's what it looks like internally, with 10 heaps now (instead of 48) and much lower gen 0 limits. Only the *actual* long-lived cache items are sticking around, i.e. things we actually need memory for \o/:
@maoni0 If you're curious how to quickly analyze memory dumps - there are several tools out there. I wrote a quick LinqPad script a while back to answer common questions we care about. It's in a gist here: gist.github.com/NickCraver/d62…
Take a memory dump (e.g. task manager) and have some fun!
Take a memory dump (e.g. task manager) and have some fun!
Added a small PR for updating the GC docs in hopes it helps the next person: github.com/dotnet/coreclr…
