llama.cpp replaces a transpose-plus-sum_rows indexer reduction with strided views, worth 9% prefill on an RTX PRO 6000

GitHub·medium signalXBlueskyLinkedInCopy link

Build b10730 (#28023), published 2026-09-01, rewrites the qwen4exp indexer head reduction that went through a transpose and a sum_rows over ne[1], leaving sum_rows with ne0 = 4 — one block per row for a four-element reduction — while the transpose copied the whole block-by-token surface twice on the way in. Because the heads are adjacent on ne[1], each is now a strided view and the sum is a short chain of adds. On an RTX PRO 6000 with Qwen3.8-Flash-Next UD-Q4_K_XL, fa on and 55k context, prompt processing goes from 2,170 to 2,366 t/s with generation unaffected, and the gain grows with context and ubatch size.