A Mamba block-allocation miscount was stalling entire vLLM servers serving Kimi K3 from external KV storage

GitHub·medium signalXBlueskyLinkedInCopy link

PR #57050 fixes get_num_blocks_to_allocate miscalculating the physical blocks a long request needs when loading prefix cache from external KV storage such as Mooncake, so the request is refused admission and sits in the waiting queue forever, stalling the whole system. The maintainers trace it as a likely regression from #53614. The reproduction is an 8 x B300 Kimi-K3 serve with --decode-context-parallel-size 8, --max-model-len 1048576 and TOKENSPEED_MLA, which is a narrow enough configuration that most operators would have hit it as an unexplained hang rather than an error.