Sam Rose: 'Quantization from the Ground Up' — Best Visual Explanation of LLM Compression with Interactive Benchmarks

Simon Willison·medium signal

Sam Rose published an interactive essay explaining LLM quantization, featuring the 'best visual explanation' of binary float32 representation and how outlier 'super weights' (Apple's term) can cause models to output complete gibberish if removed. Includes hands-on benchmarks using llama.cpp perplexity tool and GPQA on Qwen 3.5 9B, concluding that 16-bit to 8-bit carries almost no quality penalty while 16-bit to 4-bit retains ~90% quality. Rose called it 'the best post I've ever made.' Highlighted by Simon Willison.