Dynamic Quantiser makes exact-size mixed-precision GGUFs with no calibration data by minimizing cosine deviation

r/LocalLLaMA (36 upvotes)·low signalXBlueskyLinkedInCopy link

The open-source tool picks a quantization level for each tensor to minimize whole-model cosine deviation at a target file size. The author reports 0.99 Pearson correlation between that metric and KLD. It needs only a bf16/f16/f32 GGUF and llama.cpp, and after an initial table is built each new solution takes seconds. The author says it beats standard Qx_K_M quants but trails Unsloth's dynamic quants on pure text (r/LocalLLaMA, 36 upvotes). A commenter pointed to ExLlamaV3 as getting lower KLD at the same size. It is most useful for VRAM-constrained users who need a quant sized exactly to their card.