Execution Ledger Lifts SWE-bench Pass Rate 56% to 64%

A runtime wrapper called Ledger raised GPT-5 mini's SWE-bench Verified pass rate from 56.2% to 64.2% while cutting total cost 28.9%, according to a paper posted to arXiv.

The result isn't a cherry-picked subset. It's all 500 SWE-bench Verified instances, and the gain held on a second model: MiniMax M2.5 went from 75.8% to 81.0% with cost down 31.8%. Plugged into OpenAI Codex, Ledger added 3.4 points at 24.4% lower cost.

Ledger doesn't touch the agent itself. It tracks what's been observed, modified, and attempted across a session. Then it acts on that record at two points.

The inform path appends a compact state summary to the prompt. The govern path checks each proposed command against the ledger. If a result's still valid, it hands that back instead of letting the agent re-run the command, and flags the repeat. Neither path costs an extra model call.

The paper's ablation splits the credit cleanly: govern drives most of the accuracy gain, inform drives most of the savings. That's useful if you're deciding which half to build first.

Context spending, done wrong everywhere else

A companion paper tested prompt wording across six models. Asking a model to develop and compare several approaches alone multiplies reasoning-token spend 2.4 to 7.4 times. A bounded prompt, stating scope and a stop condition, is cost-neutral and can cut reasoning roughly in half.

Wix's Helpmate found a similar pattern in skill selection. Filtering by what an account can actually execute, rather than by relevance, cut context 90.5% across 756,600 messages. Replaying 1,000 risk-enriched conversations with everything exposed, the model picked a production-blocked skill 7.8% of the time.

I'm building the govern path next.