An audit of 443 GGUF quants found 64 that cannot be the quant their filename claims

A builder pulled 443 GGUF quantizations across 25 Hugging Face repos and checked whether each file's bits-per-weight matched the type in its name. 64 of them didn't. (r/LocalLLaMA)

The mechanism is clean, which is what makes it bad. K-quants and i-quants need the first tensor dimension divisible by 256. When it isn't, llama-quantize doesn't error. It silently substitutes a compatible 32-block type, usually IQ4_NL or Q4_0, landing somewhere around 4.5 bpw. The filename still says IQ2_XXS. So does the model card. So does the GGUF metadata. The only place the substitution appears is the quantize log, which the person who uploaded the file saw and the person downloading it never will.

On Nemotron-3.5-Lightning, all four IQ2 rungs are the same 4.58 bpw file under four different names. Four distinct downloads, four distinct size expectations, one file. Two independent uploaders reproduced it, which rules out sloppiness at any single repo and points at the tooling. The behavior has been in llama.cpp since PR #3747, filed in 2023.

I find this more interesting than the average CVE because nobody did anything wrong. The uploader ran the standard command. The tool did what it was designed to do. The warning was emitted. The failure is entirely in the gap between where the warning went and who needed to read it, and that gap is three years old and load-bearing for a large chunk of the local-model ecosystem.

The practical consequence: anyone who benchmarked a "2-bit" quant against a "4-bit" quant on an affected model was comparing two identical files and attributing the noise to quantization. Any VRAM-planning spreadsheet built on filename bpw is wrong for those 64 files. Any conclusion of the form "IQ2 is surprisingly good on this model" deserves a second look.

What to do is one command. Read the bpw off the file you're actually running, divide file size in bits by parameter count, and compare it to what the name promises. If a 2-bit quant is sitting at 4.5 bpw, you found one. And treat any local-model benchmark that reports quant type without reporting measured bpw as unverified until someone checks.

Adjacent to this, and worth pairing: an r/LocalLLaMA thread with only 53 upvotes but 74 comments argues q8 KV cache measurably hurts Qwen3.8-27B output quality rather than being the near-free win it's usually described as. (r/LocalLLaMA) The comment-to-score ratio is the signal there, since it's the shape that marks contested claims on that sub. Two threads in one day both saying the same thing about local inference: the defaults everyone repeats have less measurement behind them than you'd assume.