Swapping the N-gram layer of an IQ4 Qwen 3.8 Next quant to Q8 costs essentially no decode speed

r/LocalLLaMA·low signalXBlueskyLinkedInCopy link

Following a report that someone on a 5090 replaced the N-gram portion of a Qwen 3.8 Next UD-Q4 model with BF16, a user tried the same trick at Q8 on an IQ4_XS quant and measured decode before and after at 8.82-8.83 t/s versus 8.81-9.37 t/s, so no meaningful degradation. The N-gram block is about 51B of the model, which is why raising just that component's precision is a cheap targeted upgrade compared to moving the whole quant up a tier. Output quality improvement is still untested by the author, so this is currently a speed-neutrality result rather than a quality result.