SGLang brings HiSparse hierarchical sparse KV cache to Intel XPU and makes the gaps fail loudly
PR #32792 (merged 2026-09-21) enables HiSparse on Intel XPU alongside CUDA and ROCm by registering XPU in `ALLOC_MEMORY_FUNCS` (using torch's `pin_memory=True` rather than `cudaHostRegister`) and widening three `_is_cuda or _is_hip` import gates, with the two hot kernels AOT-compiled into the `sgl_kernel` wheel instead of JIT-compiled. Top-k page swap-in, device-buffer LRU across decode steps, batched multi-request swap-in with padding and PD decode host-slot preallocation are verified. The part worth copying is how the unsupported pieces behave: shared-index prefetch is bound to a raising stub and disabled at init with a warning, and `SGLANG_DEBUG_HISPARSE_SKIP_IO` now raises rather than silently reporting timings that include the KV copy.
Source
↳ Follow the thread