SubtleMemory Benchmark: Long-Horizon Agents Are Weak at Telling Complementary From Contradictory Memories

HuggingFace Daily Papers·medium signal

SubtleMemory (arXiv 2606.05761) targets a gap in long-term memory evals: as persistent assistants accumulate related memories, correct behavior depends on memory relations (complementary, nuanced, contradictory), not isolated recall. With 1,522 evaluation instances over 10 long histories built from 1,090 relation-controlled memory-variant sets, it finds that six standalone memory systems plus native and plugin agent memory modules all remain weak at fine-grained relational discrimination. For builders of memory-augmented agents, it pinpoints retrieval-vs-reasoning failure stages with diagnostic protocols.