Fetching from the wire…
Public story · 2026-09-08 · high
The fix runs the 20-iteration Sinkhorn algorithm in registers, cutting per-token dispatches from about 16,000 and matching DeepSeek's reference code to 6e-7.
Why now: Vulkan was the last major llama.cpp backend still missing these fused ops, so this PR closes a gap every other backend already had covered.
llama.cpp's Vulkan backend closed its last major gap in DeepSeek-V4 decoding speed. On DeepSeek-V4-Flash, the unfused hyper-connection math was burning about 32% of decode op time on gfx1151, the GPU inside AMD's Strix Halo APUs, spread across roughly 16,000 dispatches per token.
llama.cpp PR #26578 adds three fused kernels, DSV4_HC_COMB, DSV4_HC_PRE and DSV4_HC_POST, for the Sinkhorn comb chain DeepSeek-V4 runs during decode. Vulkan was the last major llama.cpp backend still running that chain unfused, one dispatch at a time.
The fused shader runs the full 20-iteration Sinkhorn algorithm inside GPU registers, using subgroupShuffleXor to move data within 16-lane blocks, skipping the round trip to memory for intermediate results. What used to take about 137 strictly ordered node executions per site now runs as a single dispatch. The author checked the output against a float64 reference and against DeepSeek's own modeling_deepseek_v4.py code, matching to about 6e-7.
Strix Halo owners running DeepSeek-V4 through llama.cpp's Vulkan backend should update. It doesn't change what the model outputs, only how long the wait is for it.
Each link below shares sources, entities, or timing with this story.
DeepSeek-V4-Flash-0731 landed July 31 under MIT with a DSpark speculative-decoding module attached. Terminal Bench 2.1: 82.7. Toolathlon-Verified: 70.3. DSBench-FullStack: 68.7. DeepSWE: 54.4. NL2Repo: 54.2. The model card claims it beats DeepSeek-V4-Pro (Preview) "despite its...
DeepSeek released V4 on April 24 and the numbers demand attention. V4-Pro is 1.6 trillion parameters total with 49 billion active, MIT-licensed, native 1M-token context. It scores 80.6% on SWE-bench Verified, putting it within 0.2 points of Claude Opus 4.6. On Terminal-Bench 2...
The leaderboard says first place. The methodology says you should check your own bill. Qwen3.8 Max now ranks first on Artificial Analysis' agentic index, scoring 86.1 on OSWorld-Verified ahead of GPT-5.6 Sol Max at 83.2 and Fable 5 at 85.0, priced at $2.00/M input and $6.00/M...
For about a year, "run your agent locally" meant accepting a model that couldn't reliably call a tool twice in a row. That excuse is gone. Meta Superintelligence Labs published Muse Glimmer today: a 29.6B dense causal transformer, 52 layers, 6,656 hidden dim, with a ~1.8B ViT-...
DeepSeek released V4 Preview on April 24 with two open-weight variants: V4-Pro (1.6T total parameters, 49B activated via MoE) and V4-Flash (284B parameters, 13B activated). Both support 1M-token context windows. Both are Apache 2.0 licensed. Both are live right now on Hugging...
DeepSeek dropped V4 in mid-June as an open-weight model with a 1-million-token context window, priced at $1.74 per million input tokens, posting near-parity with GPT-5.4 on math and Q&A benchmarks (MindStudio). That's the headline number. The architecture underneath is more in...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.