Dispatch
Hugging Face used GRPO to teach a 35B coding model to paint watercolours, and published the whole environment
A September 3 post trained Qwen/Qwen3.5-35B-A3B with LoRA using GRPO inside OpenEnv, which wraps a JavaScript rendering pipeline (the p5.brush library, headless Chromium, and validation gates rejecting invalid sketches) as an RL environment. Three runs of 60-110 steps, 8 rollouts per step and 240 episodes moved mean group reward from 0.45-0.58 to 0.71-0.82, on one H200 for 18-34 hours per run. The interesting mechanic for anyone designing rewards is the ordering the model learned: it eliminated invalid outputs first, and only improved quality once the pairwise style judge carried enough weight.
↳ Follow the thread