vLLM was writing speculative-decode KV into other requests' prefix-cache blocks and poisoning them with NaN

GitHub·high signalXBlueskyLinkedInCopy link

vLLM PR #56734 (merged 2026-09-21) fixes KV-cache corruption in Model Runner V2 spec decoding under data parallelism: an idle DP rank's dummy batch ran the drafter's multi-step decode through the persistent per-slot block tables with `idx_mapping = arange(num_reqs)`, writing drafter K/V into offsets 1..k-1 of the first block of whatever request last held that slot. On GLM-5.2 across 4x GB200 with MTP k=5, DP=4 EP and fp8 KV, the symptom was exactly-zero MTP acceptance (p0 = 0.0000, acc_len = 1.0) that latched onto a conversation and cleared only after `POST /reset_prefix_cache`; dumped rows were fp8 0x7F NaN. The guard had to live in `_compute_slot_mappings_kernel` rather than Python because the fused multi-step path recomputes slot mappings inside the captured CUDA graph.