A Small RL-Trained Concept Generator Raises a Larger Frozen Model's pass@k Over Repeated Sampling and Transfers Across Model Families

arXiv·medium signalXBlueskyLinkedInCopy link

Labiad, Kowalski and Schoenauer argue that repeated sampling explores only through decoding noise and produces near-duplicate attempts. They first sample problem-specific concepts or strategies, then condition answers on them, and train a small concept generator with RL to maximize a frozen larger model's success. On hard math problems the trained generator improved pass@k at the same answer budget, beat concepts from much larger untuned models, and transferred to answer models it never trained against, including one from another family. The result is a cheap, reusable search policy you can put in front of an existing model.