Latest Exact Match Attention Keeps Generation Speed Constant With the Dictionary in RAM, Not VRAM, and Out-Recalls Gated DeltaNet
arXiv·low signal
LEMA binarizes queries and keys so each query attends only to the latest exactly matching key. Brösamle proves a two-way correspondence with word-RAMs in which cost per token is independent of context length. Models up to 834M parameters matched softmax transformers about half their size on loss and recalled across longer distances than same-size Gated DeltaNet. Dictionary-based inference gave constant generation speed comparable to GDN, with the growing state held in main memory. Code is on GitHub.