A vLLM sparse-attention perf PR reports its own result as effectively flat

GitHub·low signalXBlueskyLinkedInCopy link

PR #57885 (merged 2026-09-21) removes four validity copies and one unused division per decode step from vLLM's sparse attention metadata preparation, by writing SWA token validity directly into its persistent buffer and computing compressed prefill sequence lengths only when the batch actually contains prefills. On DeepSeek-V4.1-Flash across 4x GB200 at TP4 with FlashInfer sparse attention, mean TPOT moved 4.467 ms to 4.457 ms and median ITL 4.383 to 4.365 ms — and the author states plainly that mean TPOT is effectively flat and claims no material end-to-end speedup or tail-latency improvement. Notable as a counter-example to benchmark inflation: the change is justified by removing work, not by a number.