vLLM adds backpressure detection to KV offloading with an EMA and 1.0s/0.5s hysteresis

GitHub·high signalXBlueskyLinkedInCopy link

PR #50045, merged 2026-09-18, stops slow secondary KV tiers (disk, shared storage, P2P) from accumulating unbounded job queues. An exponential moving average tracks per-tier store completion latency; a tier enters pressure when the EMA passes a 1.0s high-water mark and clears below 0.5s, and stores to a pressured tier are dropped while loads keep working. Two Prometheus metrics per tier, vllm:kv_offload_tiering_backpressure_active and vllm:kv_offload_tiering_backpressure_stores_dropped, make the drop visible instead of silent.