PPL-Factory Picks Fine-Tuning Data Under a Fixed Budget

PPL-Factory scores fine-tuning samples by perplexity and picks them to fit a fixed compute budget, per a preprint from McGill's Zhang and Gross. The preprint frames that as the realistic setup: rationed compute, not an unlimited corpus and an unlimited GPU pool. Selection under that kind of ceiling is the actual problem, not a side case.

The method is both task-aware and budget-aware. Zhang and Gross test it across plain language modeling and reasoning fine-tunes, per the preprint. Reasoning data behaves differently enough from general text that testing both is the harder claim to defend.

A sample can score as informative under perplexity and still be a bad example for teaching a model to reason. Surprising isn't the same as right. That's a gap the language modeling result alone wouldn't expose.

The preprint summary doesn't say how much compute the method saves against full-dataset fine-tuning, or which base models Zhang and Gross ran it on.