MemPalace retires its own 100 percent memory benchmark

MemPalace calls itself the best-benchmarked open-source AI memory system, and with 57,551 GitHub stars it's one of the leading local-first memory layers builders reach for. Its headline claim was a perfect 100% score on LoCoMo across all five question types.

That number didn't hold up. The test ran at top_k=50, a retrieval setting that pulls back effectively the entire corpus. When you hand a model the whole document set, there's no retrieval happening, no memory system doing work. You're just measuring whether Sonnet can read and answer questions, which it's good at regardless of what's storing the data.

Someone caught this and said so publicly. The maintainer didn't argue or bury it. They acknowledged the critique and pulled the disputed numbers.

What's left after the retraction

The honest figures are 88.9% R@10 on hybrid v5 without reranking, and 96.6% Recall@5 in the zero-API-cost category. Those are real, defensible numbers for a memory layer, just not headline-grabbing ones.

This is the part worth sitting with. Inflated benchmarks in AI tooling are common enough that most of us have stopped being surprised by them. What's genuinely rare is a maintainer retracting their own claim once it's challenged, instead of quietly editing the README or going silent. If you're evaluating memory systems for a project, check what retrieval setting a benchmark actually used before you trust the top-line score. top_k=50 against a small eval corpus isn't a memory test. It's a reading test wearing a memory test's clothes.