The Best Explainer Yet on Engrams Argues They Buy Depth, Not the Ability to Run 1T Models on an SSD

r/LocalLLaMA·medium signalXBlueskyLinkedInCopy link

A 1,134-upvote r/LocalLLaMA post pushes back on the claim circulating since Qwen3.8-Flash-Next that n-gram tables let you run 1T+ models with 980B parameters offloaded to SSD. The argument: an Engram is an embedding table keyed on the last two or three tokens rather than one token ID, so 'New York' gets its own memorized vector via an O(1) hash lookup with no FLOPs, which frees early transformer layers from re-deriving multi-token entities and spends that depth on reasoning instead. Commenters report the practical tell is improved letter counting with minimal reasoning and better negation handling, and note that because lookups cost no compute you can quantize the weights to Q4_K_XL while keeping the engram table at native precision.