FlashPrefill V2 Reports 47x Prefill Speedup Over FlashAttention-2 at 128K Context and Ships as an SGLang Backend

arXiv 2608.19758·high signalXBlueskyLinkedInCopy link

FlashPrefill V2 turns the earlier block-sparse prefill prototype into something deployable: a mean correction term suppresses approximation error at extreme sparsity, the operator is rebuilt with PackGQA memory access, warp specialization and pingpong pipelining to align with FlashAttention-3/4, and FP8 inference is supported. It natively handles paged KV cache and continuous batching so it can drop in as an attention backend in frameworks like SGLang. On NVIDIA H20 GPUs it reports up to 47.26x and 27.19x speedups over FlashAttention-2 at 128K context in FP8 and BF16, and still 30.49x over an FA3/4-aligned dense baseline in FP8.