vLLM found six identical GPU kernel launches per step

vLLM's build_attn_metadata() was launching one token-to-request mapping kernel per KV cache group, even when every group in the same call saw identical query boundaries. A trace on DeepSeek V4.1 spec decoding caught it: 6 launches during target prep and 3 during draft prep per step, all computing the same mapping over and over.

The fix, in PR #57102, adds a three-line per-call cache. Total kernel launches dropped from 30 to 25 in one path and 12 to 10 in the other. Host wall time per call fell 11.1% and 12.3% on GB200.

The stakes are narrow but real. Kernel launch overhead is fixed cost that doesn't scale with model size, so it eats a bigger share of the budget the smaller and faster your decode step gets. Spec decoding is exactly that case: short, frequent calls where launch overhead can rival the compute itself. Six redundant launches a step adds up fast at production request volumes.

A separate change, PR #49942, adds CPU FP8 W8A8 quantization for dense and MoE models on Intel DMR hardware. It's validated on DeepSeek-R1 and Qwen3.5-35B-A3B-FP8, with a gsm8k score of 0.802 on the quantized model. That score puts a 35B-parameter MoE model within reach of CPU-only serving, no GPU required.

Neither PR claims to be finished work. The kernel cache fix doesn't say whether the same redundant-launch pattern exists elsewhere in the attention metadata path, and the CPU FP8 numbers cover two models, not a broader sweep. If you're running spec decoding at scale, the launch-count math is worth checking against your own trace before assuming it applies.