vLLM Ships Day-0 Kimi K3 Support with a Rebuilt Prefix-Cache Model for Kimi Delta Attention
vLLM Blog·high signal
vLLM's preview post details why KDA broke their caching assumptions: instead of per-token KV pairs, KDA keeps a recurrent state matrix plus convolution state updated per token. Their fix separates three previously-conflated concepts — physical block size (GPU allocation), scheduler alignment (execution stop points), and prefix-match granularity — enabling copy-on-write partial cache hits inside larger physical blocks. Ships with fused KDA decode kernels and MXFP4 MoE optimization on NVIDIA, a FlyDSL MLIR stack with A16W4/A8W4 operators on AMD, and prefill/decode disaggregation via Mooncake.