Delaying ready agent turns cuts P95 latency 3.5x

A new scheduling method holds back agent turns that are ready to run instead of releasing them right away, cutting P95 latency by up to 3.50x under contention, per a paper posted to arXiv. Most agent runtimes release a turn the moment it's ready.

Under load, that released but unfinished work piles up somewhere a workflow can no longer reorder it. That backlog is the tail latency anyone running several agents at once eventually hits.

The method jointly decides which ready turn to release and how much unfinished released work to allow at once.

It scores that decision with a mean-CVaR objective over tail risk, using online estimates of remaining turn work.

A budget on queue pressure caps how much unfinished work can sit released at any moment. Tested on real agent traces from software tasks across multiple models, the method matches eager release when load is light. The gap opens only once turns start competing for compute, where it delivers the 3.50x P95 improvement.

Model speed isn't the constraint here. The default of releasing every ready turn immediately is, and it backs up as soon as several agents compete for the same compute. What the paper doesn't say is whether the mean-CVaR scoring adds enough overhead to matter at small scale, or whether it holds up past the software-task traces it was tested on.