Dispatch
100 GRPO steps on a free Colab GPU lifted a 350M model's JSON compliance from 18.0% to 31.9%
A Hugging Face post published September 3 fine-tuned LiquidAI/LFM2.5-350M with TRL's GRPO on roughly 500 samples from nvidia/Nemotron-RL-instruction_following-structured_outputs, augmented with 40% code-block instructions and 20% top-level-array conversions. Overall IFStruct pass rate went from 22.6% to 29.7% in 100 steps, with JSON format specifically going 18.0% to 31.9%, on a free-tier Colab or Kaggle GPU with 16GB VRAM. The reward is three functions, JSON validity, field count accuracy and schema validation, weighted [1.0, 0.5, 2.0], which is a directly copyable recipe for making a small local model reliable at structured output.
↳ Follow the thread