Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-10 · high
A Tsinghua-led paper finds multi-stage reinforcement learning does the opposite, ending 24.9% above the base model.
Why now: The paper's mechanism for the SFT-versus-RL split is covered in the August 10 briefing.
Sequential fine-tuning drags models 23.1% below their base performance while sequential reinforcement learning pushes them 24.9% above it, per a Tsinghua-led paper posted on Hugging Face.
The gap matters for anyone chaining task-specific fine-tuning runs across math, coding, logic, and science on one model. Standard sequential fine-tuning can leave it worse than where it started.
The paper ties the split to how each method moves parameters. RL updates run roughly 100x smaller in magnitude than SFT updates. Gradients across different RL tasks share about 10⁻⁵ similarity in parameter space, versus roughly 10⁻¹ for SFT tasks.
That gap means RL tasks barely interfere with each other during training, while SFT tasks overwrite each other's progress as they're chained.
The paper's fix trains a separate model per task with RL, then merges the results, in a recipe it calls Parallel-RL. That approach keeps 102.8% of the performance a single-task RL model would hit on its own, using only 5% additional adaptation to merge.
That mechanism should hold outside this paper's four benchmarked domains, not just within them. Worth checking whether the 100x gap in update magnitude survives on models larger than the ones tested here.
Each link below shares sources, entities, or timing with this story.
Modular released Mojo / Shared entity: Modular / Earlier coverage
Linked by a graph relationship (Modular released Mojo); both cover Modular; earlier Modular coverage from 2026-06-26.
Modular built by Chris Lattner
Linked by a graph relationship (Modular built by Chris Lattner).
Linked by a graph relationship (Modular built by Chris Lattner).
Linked by a graph relationship (Modular built by Chris Lattner).
Shared entity: SFT / Shared topic / Earlier coverage
Both cover SFT; overlapping topics (behind, model); earlier SFT coverage from 2026-08-07.
Both cover SFT; overlapping topics (fine tuning, model); earlier SFT coverage from 2026-07-22.
Shared entity: Multi / Shared topic / Earlier coverage
Both cover Multi; overlapping topics (coding, model); earlier Multi coverage from 2026-05-24.
Both cover Multi; overlapping topics (coding, model); earlier Multi coverage from 2026-05-22.