AgentSysBench measures agentic workloads and finds non-LLM components dominate latency in half of apps

arXiv·high signalXBlueskyLinkedInCopy link

A systems-level benchmark suite of ten agentic applications with unified instrumentation finds six properties that break conventional LLM-serving assumptions: non-LLM components dominate latency in 5 of 10 apps, sandbox working sets peak at 28 GB per session, task latencies across GPU-bound inference, memory-bound retrieval and CPU-bound sandboxes diverge by up to 32x, and production sessions sit idle for minutes to hours between active steps. Four design fixes follow directly: task-aware serving cuts latency 29-40%, communication-aware placement up to 4.5x, state offloading cuts memory 4.6x, and tool-result caching removes 35.2% of redundant search calls. It also names a 'control-plane tax' where auxiliary LLM calls and tool-schema context crowd out productive compute.