EXL3 quant fits a 30B model in 12GB with 100K context

A LocalLLaMA user got Muse Glimmer 30B running fully in 12GB of VRAM on an EXL3-SC quant at 3.00bpw H4, with a Q8 KV cache. The model's official K-quant needs 17GB, more than a 12GB card holds, with no noticeable quality drop against it for agent work.

A LocalLLaMA thread on EXL3 quants has the numbers: 100K tokens of context at around 30 tokens a second, all inside that 12GB budget. A 17GB model that doesn't fit on a 12GB card normally means spilling to system RAM or dropping to a smaller model instead. EXL3's 3.00bpw format skips both trade-offs.

The same tester tried Qwen 3.8 27B at a more aggressive SC2.20bpw H3 quant and called it usable. For coding, they still went back to Unsloth's UD_Q4_K_XL, a GGUF quant.

The post doesn't name the hardware beyond the 12GB VRAM ceiling. It has no formal eval score, just one person's read against their daily-driver quant.