Tools
IQ3_S MMQ kernels land in llama.cpp's Vulkan backend, up to 1.51x prefill on Intel Arc
PR #28822 (merged 2026-09-18 08:46 UTC) adds IQ3_S MMQ matmul kernels for Vulkan, because llama.cpp avoids VK_KHR_cooperative_matrix on Intel graphics under Linux and was falling back to dequantize-to-float matmul. On an Arc A770 16GB with Mesa 26.0.8, Qwen3.6-35B-A3B-REAP-48-v2 (90% IQ3_S tensors) went from 157.9 to 239.2 t/s at pp64 (1.51x), 481.4 to 654.9 at pp512 (1.36x) and 632.7 to 797.8 at pp2048 (1.26x), with token generation flat. The submitter found it because the same model's v2 i-quant release was far slower than its v1 k-quant release.
Source
↳ Follow the thread