Tools
llama.cpp traces Qwen3-tts NaN output to an F16 FFN weight overflowing on a 1.5e5 activation peak
Build b10760 (2026-09-02, PR #28231) keeps the Qwen3-tts code predictor's `ffn_down` in F32 because that layer's intermediate activation peaks around 1.5e5, well past F16's 65504 ceiling. Since mul_mat casts its input to the weight type, an F16 `ffn_down` turned that peak into inf, the residual followed, and the next rms_norm produced NaN. The fix cites a reference forward pass in float32 giving 145109 against 145396 measured in the graph. It is a clean worked example of why blanket quantization of every weight in a model breaks specific layers.
Source
↳ Follow the thread