An open-source llama.cpp branch hit ~1,204 t/s prefill on a 177B model running on one Ryzen AI Max+ box

pwilkin.github.io (via r/LocalLLaMA, 190 upvotes)·medium signalXBlueskyLinkedInCopy link

ilintar published the full optimization writeup at pwilkin.github.io/strix-halo after a closed-source server called Halogen claimed 1.2k t/s prefill on Qwen3.8 Flash Next while the community llama.cpp fork was stuck near 400. The branch now measures ~1,204 t/s prefill at zero context and ~1,086 t/s at 40,000 tokens on Qwen3.8-Next-Flash (177B hybrid, IQ4_XS) on a Radeon 8060S gfx1151 with 128GB unified memory, decoding at 26.28 ± 0.29 t/s. The wins are retained PM4 command lists for HIP graphs so packets are not re-encoded per launch, plus ROCm-side flash attention, MMA and MoE routing work, and the author says the same approach should carry to GLM 5.3 Flash's sparse attention.