Full-pipeline FP8 RL collapses from quantization noise distorting the importance ratio, and Calibrated Clipping fixes it
arXiv / HuggingFace Daily Papers·low signal
arXiv 2609.22870 (19 Sep) traces mid-training entropy surges and garbled outputs in FP8 RL to compounded quantization noise. The noise pushes negative-advantage tokens outside the trust region and zeroes their gradients, so bad outputs stop being penalized. Calibrated Clipping matches FP8 clipping bounds to BF16 distributions. It is relevant to anyone trying to run cheaper RL fine-tunes in FP8.