llama.cpp fixes a SYCL path that moved 4.56 GB per 2048-token prefill chunk by staging the KV cache instead of binding it

GitHub·medium signalXBlueskyLinkedInCopy link

Build b10669 (2026-08-28) binds the f16 KV cache in place for the oneDNN SDPA path. The commit message does the arithmetic on Qwen3.8 27B Q4_K_S at a live KV length of 34,816: 71.3 MB per tensor, 142.6 MB staged per call for K and V, 285.2 MB of traffic per call, and 16 calls per ubatch equals 4.56 GB of memory traffic for a single 2048-token prefill chunk, scaling with live KV length (the first ubatch at seq=2048 moves only 0.27 GB). Build b10668 the same day adds Metal flash-attention vector tunings for M3 Max, M5 and M5 Pro, contributed by community members running the repo's own `ggml-metal-tuning` sweep.