vLLM Ships Async Speculative Decoding with Zero-Bubble Overlap + Gemma 4 Full Support

vLLM GitHub·medium signal

vLLM's latest release adds async scheduling for speculative decoding with zero-bubble overlap — significantly improving throughput for production serving. Also ships full Gemma 4 support (MoE, multimodal, reasoning, tool use), piecewise CUDA graphs for pipeline parallelism, full CUDA graph capture for vision encoders, and a general CPU KV cache offloading mechanism with pluggable policy. V1 engine shows up to 24% throughput improvement over v0.