r/LocalLLaMA Reverse-Engineered Qwen4's N-Gram Architecture: MoE Does Reasoning, N-Grams Do Recall, and Only RAM Works

r/LocalLLaMA·medium signalXBlueskyLinkedInCopy link

A 217-upvote r/LocalLLaMA writeup breaks down the Qwen4Exp architecture behind Qwen3.8-Flash-Next: a router picks experts late in the layer with a large payload, so experts resist offload, while an n-gram table is a hash lookup known early and cheap to fetch. The author's conclusion is that roughly 25 percent of weights can move to n-gram before the tradeoff inverts, turning a 176B model into 125B in RAM plus 51B elsewhere. The thread's correction to the original SSD claim matters for builders: the n-gram index is hashed, so access is random and RAM is the only viable tier.