Crossflow Makes Prefill-Decode Disaggregation Elastic and Lifts Token Throughput 16-17% Over Static P/D on Agentic Traffic
arXiv·medium signal
arXiv 2609.27085 (22 Sep) measures uncached-input-to-output ratios with peak-to-mean swings up to 4.7x at minute scale in a production fleet, and a 24.5x median hourly range in a public agentic trace. Static P/D pools sized at p95 waste up to 17% of capacity. Crossflow has decode nodes publish short-lived revocable leases for local prefill, improving throughput 16.2-17.4% (geomean) and up to 43.4% at high load while lowering mean TTFT at every point tested.