llama.cpp adds recurrent state rollback for Qwen3.8-Flash-Next, taking MTP speculative decoding from 123 to 183 tok/s

GitHub·medium signalXBlueskyLinkedInCopy link

Build b10731, published 2026-09-01, implements recurrent state rollback for qwen4exp (#28123), which MTP speculative decoding needs so the target state can move back by the number of rejected draft tokens. Without it the context was classified SEQ_RM_TYPE_FULL and the server serialized the whole recurrent state to host memory every round, costing more than the drafting saved. Measured on Qwen3.8-Flash-Next UD-Q4_K_XL with n-max 3 and a single slot, decoding reaches 183 tok/s on code and 144 on prose, against 123 and 83 on the same branch before the change and 108 tok/s with no draft at all.