vLLM 0.29.0 makes Model Runner V2 the default

vLLM released version 0.29.0, and Model Runner V2 is now the default execution path. CUDA graph memory profiling auto-sizes the KV cache instead of relying on a fixed allocation. Paired with batch-sharded sampling, per-step logits memory drops by a factor of 1 over tensor-parallel size, so a TP-8 deployment holds an eighth of the logits memory the sampler used to need.

The release notes list 594 commits from 277 contributors, 91 of them first-timers to the project.

Model Runner V1 isn't gone entirely, but it's down to life support. It only survives for a handful of ROCm models now that MRV2 is the default everywhere else. FlexOlmo, Olmo3, and Hunyuan move to the Transformers backend in this release, and the PyAV video decoder is removed outright. Ten architectures got cut in total.

This release also nudges everyone off the old server entrypoint. Running python -m vllm.entrypoints.openai.api_server directly is deprecated in favor of the vllm serve command. It still works for now, but treat it as a signal to update deploy scripts before it stops working.

FlashInfer all-reduce is on by default now for tensor-parallel CUDA groups. Anyone whose multi-GPU throughput numbers move after upgrading can opt out with VLLM_ALLREDUCE_USE_FLASHINFER=0.

How long MRV1's ROCm carve-out lasts isn't stated, so anyone on AMD hardware should watch for the release that finishes the migration.