Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-24 · high
A structured supersession memory called MemStrata found the current answer 91% of the time, at the same speed as plain retrieval.
Why now: These results entered the record on August 24, 2026, drawn from 707 real GitHub issue histories rather than synthetic benchmarks.
Plain retrieval-augmented generation returns an outdated, already-fixed value 36 to 38 percent of the time on real GitHub issue histories, per a paper testing RAG against real code fixes. That 36-38% failure rate matters for any coding agent that leans on RAG to remember what changed in a codebase. It can retrieve the right file and still hand back a stale fact, since ranking by similarity doesn't track which version is current.
The failure cases come from SWE-bench Lite and Verified. An LLM reranker layered on top of retrieval didn't help either, adding about 18 seconds of latency against 2.1 seconds for plain retrieval, with accuracy holding at the same 0.57 to 0.59 range.
A structured memory called MemStrata replaced plain retrieval and reached 0.91 accuracy, running at the same speed as plain RAG. It stores facts as (subject, relation, object) triples and tracks which one supersedes the last, so newer facts overwrite old ones instead of competing on similarity score alone.
The authors are upfront about the limits of their own test. Only about 18% of real fixes are clean atomic transitions like the 130 cases they measured, which leaves about 82 of every 100 real-world fixes, the messier, multi-step kind, still untested.
Each link below shares sources, entities, or timing with this story.
Simon Willison released LLM / Shared entities / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover GitHub, SWE, Verified; earlier GitHub coverage from 2026-04-23.
Simon Willison released LLM / Shared entity: RAG / Same source domain / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover RAG; reported by the same outlet (arxiv.org).
LLM uses OpenAI / Shared entities / Earlier coverage / Tension
Linked by a graph relationship (LLM uses OpenAI); both cover LLM, RAG; earlier LLM coverage from 2026-07-27.
Simon Willison released LLM / Shared entities / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover Lite, SWE; earlier Lite coverage from 2026-03-17.
LLM uses OpenAI / Shared entities / Earlier coverage
Linked by a graph relationship (LLM uses OpenAI); both cover Lite, LLM; earlier Lite coverage from 2026-07-31.
Linked by a graph relationship (LLM uses OpenAI); both cover RAG, SWE; earlier RAG coverage from 2026-04-24.
Simon Willison released LLM / Shared entity: SWE / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover SWE; overlapping topics (called, didn).
Simon Willison released LLM / Shared entity: GitHub / Same source domain / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover GitHub; reported by the same outlet (arxiv.org).