110ms Memory Delay Makes AI Agents Repeat Actions

Agents took redundant actions 7.2 out of 12 times at 110 milliseconds of memory latency, versus zero at 100 microseconds, per a new arXiv paper on agent memory design.

That's the whole story in one number: slow memory doesn't cost an agent time so much as it costs the agent its own memory of what it already did, so it repeats itself. For anyone running agents against a hosted vector database, that shows up as wasted tokens and duplicated tool calls, not a latency number in a dashboard.

The paper moves memory read and write operations inside the agent's per-step loop, which only works if the store answers in about 100 microseconds. Recall backed up the redundant-action numbers: it rose from 0 out of 5 to between 3.6 and 4.8 out of 5 once memory ran in-process instead of over a network call.

There's a trap inside the fix, too. Pairing a fast in-process vector store with a network-hosted embedding model doesn't help, because embedding calls over the network run 200 to 400 milliseconds and become the new bottleneck. The paper's answer is a local embedder sitting next to the local store, not a local store on its own.

Memory store latency belongs in the correctness column, not the performance column: cross 110 milliseconds and the agent starts repeating work it already finished.