Memory Decoder at Scale: A 6.9B Bolt-On Memory Module Makes Pythia-410M Beat Pythia-12B Using 39% Fewer Total Parameters
arXiv 2607.27919 argues long-term memory should be a separately scalable parametric module rather than entangled with reasoning in one weight set, and backs it with a distributed Faiss indexing pipeline plus sparse batch-wise loading of kNN distributions. Pairing a 6.9B memory model with Pythia-410M lifts the average across 17 benchmarks from 29.86 to 37.34, edging past Pythia-12B's 37.24 while using 39% fewer total parameters; adding a 1.7B domain-specific memory to Qwen3 models from 0.6B to 14B gains over 9 points across three domains at every scale. The builder implication is that domain adaptation may be cheaper as a swappable memory module than as a fine-tune — and unlike RAG, the memory is parametric, so there is no retrieval hop at inference time.
↳ Follow the thread