New paper trains RL through the agent harness, not the model

Agent Lightning v1.0 flips how agentic reinforcement learning normally works, according to the paper posted at arXiv 2608.17528. The usual setup has the training engine drive the agent, run the environment loop, and compute rewards itself, which means your training code and your production code are two different implementations of "agent" that constantly drift apart.

Agent Lightning inverts that. The deploy-time harness owns the interaction loop. The trainer sits behind an endpoint proxy and only sees sequences of LLM request and response pairs. It doesn't know what tools exist or what the environment looks like. It just trains on the traffic.

That inversion isn't free. The authors list the problems it creates, retokenization mismatches between harness and trainer, sample merging, advantage calculation, loss normalization, backend scheduling, and say each one materially affects training stability. That's the part of the paper that usually never makes it into a blog post, and it's the part that separates a technique that works on paper from one that works in a real repo.

The result: about 3,500 lines of code, 6,000 training examples, and a Qwen3.5-9B model that went from 41.8% to 56.4% on SWE-bench Verified. Fourteen and a half points from a 9B model on a modest training set.

What to watch

The adoption is the real signal. The disaggregated proxy architecture has already been picked up by verl Uni-Agent, AReaL 2.0, slime, and Polar, four independent RL frameworks landing on the same design within one release cycle. That's not one lab's claim, that's an ecosystem agreeing on something.

Most people reading this aren't running RL jobs. But the pattern generalizes: put your telemetry at the LLM request boundary, not inside your orchestration code. I've rebuilt trace collection twice after wiring it into the orchestrator instead of the model client. Wire it into the client and you can swap harnesses without rewriting observability.