Thread by @deivid_tech

You can run Qwen3.6 35B-A3B at 128K context on a single 12GB GPU. The video illustrates how this actually works.

It's a 41-layer MoE: 256 experts, but only 8 fire per token (~3B active). The trick is a tuned VRAM↔RAM split in llama.cpp — not brute force:

-ngl 999 → offload all 41 layers to the GPU
-ncmoe 28 → but push 28 of those layers' experts into system RAM
(the other 13 layers keep their experts in VRAM)
--cache-type-k q8_0
--cache-type-v q8_0 → 8-bit KV cache, so 128K context fits
-fa on → flash attention

Why it works, per token:
1️⃣ Attention, the router, and the KV-cache for all 41 layers run in fast VRAM.
2️⃣ 13 layers keep their experts on the GPU; the 28 heaviest sit in cheaper system RAM.
3️⃣ For those 28, the GPU sends a tiny activation vector across PCIe.
4️⃣ The CPU wakes only the 8 experts that token routes to, computes them in RAM, and returns the result.

Dial -ncmoe to your card: higher = more in RAM (smaller VRAM), lower = more on GPU (faster). 28 of 41 is the sweet spot for 12GB at 128K.

35B-class intelligence, ~12GB VRAM.