Metis bakes memory into a 27B model's weights

Metis stores memory as parameters instead of retrieved text, per a paper posted under arXiv ID 2607.26760. That's a swing at retrieval-augmented memory, the default where an outside database gets queried at inference time instead of living in the weights. Metis-27B backs that bet with 50.82% on the NextMem benchmark, a number worth weighing against the 24.76% it manages on MemOps with no context.

Its memory blocks pair a local memory matrix with a normalization vector. Separate hyper-memory blocks compress hidden states into memory key-value pairs blended into standard attention. The memory persists as state across a conversation instead of being rebuilt from scratch at each turn, per the paper.

On LoCoMo Gold, Metis-27B scores 26.74%, and on the out-of-distribution ATM-Bench it drops to 18.56%, per the authors' own numbers. The code and checkpoints are public, but they're licensed CC BY-NC-SA 4.0, so nobody ships this in a commercial product without a separate deal.

The gap between that 50.82% and the 18.56% out-of-distribution score is the widest in the paper's own table.