Tools
vLLM ships release_kv_cache_memory() so RL trainers stop discarding model weights to free VRAM
PR #44890, merged 2026-09-17, adds LLM.release_kv_cache_memory(), a POST /release_kv_cache_memory endpoint and async variants. It discards only the kv_cache-tagged allocations, keeps weights resident on GPU, and pauses the scheduler until wake_up() restores memory. The previous workaround was sleep(level=2) followed by wake_up(tags=['weights']), which threw away the weights too; equivalence was validated on Qwen3-0.6B on an H200 with identical token sequences after release and restore.
Source
↳ Follow the thread