Agent reliability decays geometrically, and shrinking the context window makes it worse

One number predicts whether your agent finishes the task, and it isn't the benchmark score.

Shubhra Mittal's paper (arXiv 2609.01660) analyzed 10,664 trajectories across nine models spanning 1.2B to 671B parameters and found task success follows P(n) = p^n, where p is a single per-step reliability parameter and n is the number of steps. At p=0.95, which is a good agent, you get 59.9% success at 10 steps, 35.8% at 20, and under 1% at 100. The curve holds across the whole parameter range, which is what makes it useful. Model size buys you a better p. It does not buy you a different shape.

Daniel Vaughan converted this into Codex CLI session design rules on September 4, and the finding that changed my own setup is the counterintuitive one: restricting the context window does not slow degradation, it steepens it. That contradicts the advice everybody including me has been giving for a year. Trim the context, keep the agent focused, feed it less. Turns out less context lowers p, and p is exponentiated.

Run the arithmetic against your own harness and it gets bleak fast. A 40-step refactor at p=0.98 is 44.6%. The same refactor at p=0.95 is 12.9%. Three percentage points of per-step reliability is the difference between a coin flip and a waste of an afternoon, and no amount of prompt engineering at step 37 recovers what compounded away at step 6.

What follows from this is a design change, not a prompt change. Cap turns per session at a number you picked from the arithmetic instead of running until the model gives up. Checkpoint state to disk between phases so a fresh session starts at n=0 with the previous phase's output as input. That's the whole trick: you can't raise p much, so reset n. I've been doing a version of this by accident in my own pipeline, splitting phases into separate processes with SQLite in between, and the paper gives me the reason it works.

It also reframes today's harness benchmarks. The FrontierHarness Eval repo ran Kimi K3 through nine harnesses over the same 30 tasks and found a 16-point pass-rate spread on one model. Codex led at 66.7%, Claude Code matched DSH Creator at 63.3%. If harness choice moves the result 16 points on identical work, a big chunk of what you're measuring is p, and p is a property of the scaffold as much as the model.

The honest limit: p isn't constant across step types in real work. Reading a file is more reliable than resolving a merge. The geometric law is a floor-level model, and treating it as exact per-step will underpredict on easy sequences and overpredict on hard ones.