A 6.9B bolt-on memory module makes Pythia-410M beat Pythia-12B with 39% fewer total parameters.

arXiv 2607.27919 argues long-term memory should be a separately scalable parametric module rather than entangled with reasoning in one weight set, backed by distributed Faiss indexing and sparse batch-wise loading of kNN distributions. The 410M+6.9B pairing lifts the 17-benchmark average from 29.86 to 37.34, edging past Pythia-12B's 37.24. Adding a 1.7B domain memory to Qwen3 models from 0.6B to 14B gains over 9 points across three domains at every scale. Unlike RAG, the memory is parametric: no retrieval hop at inference.