Fetching from the wire…
01
02
03
04
05
06
07
08
Research2026-07-12 · source-backed
The insight: agents only act on complete responses, so cluster tokens-per-second becomes the real objective and per-token latency requirements relax. In a production trace, agent requests shared so much context that KV-cache reuse exceeded 80% of request tokens versus 54-62% for human traffic (arXiv). If you self-host inference for agents, your scheduler was probably tuned for humans and is leaving throughput on the table.
Each link below shares sources, entities, or timing with this story.
Same source domain / Shared topic / Tension
Reported by the same outlet (arxiv.org); overlapping topics (agent, cluster); pushes against this story (against).
Reported by the same outlet (arxiv.org); overlapping topics (agent, context); pushes against this story (versus).
Same source domain / Shared topic / Downstream implication
Reported by the same outlet (arxiv.org); overlapping topics (agent, human); traces where this leads (implication).
Same source domain / Shared topic / Tension
Reported by the same outlet (arxiv.org); overlapping topics (agent, context); pushes against this story (vs).
Reported by the same outlet (arxiv.org); overlapping topics (agent, becom); pushes against this story (but).
Reported by the same outlet (arxiv.org); overlapping topics (agent, objective); pushes against this story (against).
Reported by the same outlet (arxiv.org); overlapping topics (agent, context); pushes against this story (against).
Same source domain / Shared topic
Reported by the same outlet (arxiv.org); overlapping topics (agent, complete, context).