A Qwen 3.8 27B Quant Sweep Put Q8_0 at 98.9% Top-1 Agreement and Q4_K_M at 95.6%, and the Commenters Say the Benchmark Is the Problem
r/LocalLLaMA (205 upvotes, 101 comments), quants at huggingface.co/collections/AtomicChat/qwen-38-27b·medium signal
The Atomic Dynamic GGUF team published a four-way quant comparison on an RTX PRO 6000 using the same voxel island generation task: AD-Q4_K_M at 17.1GB, 95.6% top-1 vs BF16, 0.0113 mean KLD, 67 tok/s; AD-Q5_K_M at 20.2GB, 97.3%, 0.0042, 57 tok/s; AD-Q6_K at 25.0GB, 98.7%, 0.0011, 49 tok/s; and Q8_0 at 28.9GB, 98.9%, 0.0006, 50 tok/s. They recommend AD-Q6_K as the safe pick, but the top comment at 112 upvotes noted Q8 output looked consistently worse than lower quants, and the reply that got 34 upvotes argued that is a clear sign the task does not discriminate between quants at all. The useful builder takeaway is the KLD and top-1 columns, not the subjective ranking.