A 100-puzzle fine-tune moved a small model to 84.6%

A 100-puzzle fine-tune pushed Qwen 3 4B Base from 54% to 84.6% on MATH-500. The run took six and a half minutes on one H100, using zebra logic puzzles as training data, a 30.5-point jump. The method, called PCSS, is described in a Hugging Face post.

Anyone fine-tuning small open models on a limited budget should look at the cost profile here. One GPU and 100 examples produced a 30-point swing on a standard math benchmark, a fraction of the data and compute most reported fine-tuning runs use. The same run also gained 2.57 points on AIME 2025, smaller but real on a harder benchmark.

PCSS is a per-example calibrated sigmoid scaler derived from KTO. It reduces to a standard supervised fine-tuning gradient, multiplied by a scaler that decays as the model masters a given example. In plain terms, it backs off once the model already gets something right, so training effort concentrates on what it still gets wrong.

The gains shrink as the base model gets stronger, and the author says so plainly. Granite 4.1 3B gained 11.1 points under the same recipe. Qwen 3.5 9B, a larger and stronger model, gained only 3.1 points. The more capable the base model, the less the zebra puzzles have to teach it.

The post includes a reproduction notebook and a public dataset, so the 84.6% figure can be checked rather than taken on faith. It doesn't say whether PCSS holds up on tasks further from logic puzzles. It also doesn't say whether the MATH-500 and AIME gains carry over to messier, real-world reasoning tasks.