vLLM lands its first AuxOutput connector, storing MoE routed-expert IDs in KV-compatible block hashes

GitHub·high signalXBlueskyLinkedInCopy link

PR #45635 (merged 2026-09-21, 32 files, +3600/-873) adds the first AuxOutput Connector to vLLM, with routed-expert output (R3, the expert IDs picked for every token an MoE model executes) as the first supported type. The design note is the interesting part: R3 cannot be a request-local GPU buffer once prefix caching and async scheduling are on, because a cache hit skips execution, GPU slots are not stable identities, and speculative decoding executes rows that are later rejected. Storing immutable R3 blocks under KV-compatible block hashes lets KV and R3 for the same prefix be reused together; this PR ships a bounded local shared-memory backend, with distributed storage and DSA/logprobs auxiliary outputs named as follow-ups.