Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-17 · high
It scores itself with rule-based schema checks and an LLM judge, skipping the cost of retraining the agent that's already running.
Why now: The paper landed in the August 17 briefing on agent-recovery research.
A team trained a small language model to diagnose and recover from agent drift. The approach skips retraining the larger model that's already running, per a paper posted to arXiv. The system routes each failure through a four-stop recovery graph. It classifies the drift, flags the operation that caused it, weighs the risk, and decides how to recover.
Retraining a production agent's main model every time it wanders off course is expensive and slow. A specialized side model only needs four narrow judgment calls right, which makes it cheaper to iterate on than the deployed agent.
The reward signal for that side model mixes two checks. A rule-based check confirms the output matches the expected XML schema. A separate LLM acts as judge, scoring whether the recovery decision itself is correct, not just well-formatted.
Tested on the AppWorld benchmark, the trained model held its schema and made correct recovery calls at each stage, per the paper. The paper doesn't say how the approach holds up outside AppWorld, or what happens when drift compounds across multiple agent runs.
Recovery sidecars, not retrained main models, will become the default fix for agent drift in production. A small RL-trained model is far cheaper to retrain than the model actually doing the work. Cheap retraining wins once teams run agents past a single demo.
Each link below shares sources, entities, or timing with this story.
Simon Willison released LLM / Shared entity: LLM / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; overlapping topics (agent, model).
Simon Willison released LLM / Same source domain / Shared topic / Tension
Linked by a graph relationship (Simon Willison released LLM); reported by the same outlet (arxiv.org); overlapping topics (agent, model).
Simon Willison released LLM / Shared entity: LLM / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; overlapping topics (agent, model).
Simon Willison released LLM / Same source domain / Shared topic
Linked by a graph relationship (Simon Willison released LLM); reported by the same outlet (arxiv.org); overlapping topics (agent, each, model).
Simon Willison released LLM / Shared entity: LLM / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-07-27.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-08-16.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-19.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-18.