A training tweak changes how terminal agents explore

Most agent fine-tuning skips half of every training example, computing loss only on the tokens an agent writes and dropping the observation tokens it reads. A paper called ActObs adds loss on those observation tokens too, using data already collected, no extra parameters, sequence tokens or forward passes needed. The two approaches look identical right after supervised fine-tuning. They diverge once reinforcement learning starts, and the paper measures that gap directly.

On Qwen3-4B, ActObs beats the action-only baseline at every sampling budget tested on Terminal-Bench 2.0. On Qwen3-8B, the pattern shifts. ActObs gives up some pass@1 accuracy, but gains 3.4 percentage points at pass@16, and it solves more distinct tasks across the benchmark.

The explanation is about gradients, not accuracy. Action and observation gradients become orthogonal early in SFT, so they stop reinforcing each other. Training on actions alone then leaves a chunk of usable signal unused in the same batch.

For anyone training terminal agents, an SFT checkpoint that looks equivalent to a baseline can behave differently once RL is layered on top. Checking only pass@1 after fine-tuning can hide a difference that only shows up at pass@16. The paper doesn't test past Qwen3-8B, so whether the gap holds at larger scale is still open.