New Qwen3.8-27B GGUFs Claim BF16-Level AIME25 at 2.75 bpw by Learning the Quantizer and the Search Together

r/LocalLLaMA·medium signalXBlueskyLinkedInCopy link

A release post on r/LocalLLaMA (98 upvotes) publishes Qwen3.8-27B GGUFs built with two methods: GSQ (Gumbel-Softmax Quantization), post-training scalar quantization that jointly learns grid assignments and scales at 2-3 bits, and RCO (Riemannian Constrained Optimization), which assigns a quant type per tensor under a strict size budget by gradient descent on the task loss. The 3.00 bpw build (10.1 GB) matches the BF16 base on AIME25 at 100.00, lands within about a point on GPQA-Diamond (88.89 vs 89.90) and LiveCodeBench v6 (84.57 vs 85.71), and the 2.75 bpw build (9.3 GB) still hits AIME25 100.00 with a zero-shot average exceeding BF16. All three files run unmodified in llama.cpp, Ollama and LM Studio.