Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-03 · high
AuditCoder's own audit of 200 test cases found its failure locator catches the broken node only 26 times out of 60.
Why now: The paper is the coding-agent entry in the August 3 research roundup, benchmarked directly against AgentCoder rather than reported alone.
AuditCoder pairs generated code with a contract-annotated trace that ties every commitment, implementation, and piece of validation evidence to a stable identity, according to a paper posted to arXiv.
That's aimed at anyone who has to explain why a generated function does what it does, not just whether it passes. On the APPS benchmark, AuditCoder hit 82.5 to 83.0% pass@1; on ClassEval, 75.0 to 82.0%. Both numbers beat a CoT-plus-retry baseline, and both trail AgentCoder.
When a test fails, a locator tries to trace the failure back to one node in the task graph, or abstain instead of guessing wrong. Find the node, and bounded repair regenerates only that piece of the graph rather than the whole program.
The paper's own audit of 200 APPS records shows why that matters. Trace coverage held at 97.25%, so nearly every run produced a complete record. The locator, though, pinned the actual failing node in just 26 of the 60 failure cases it looked at.
What AuditCoder actually proves is that a near-complete audit trail and an accurate diagnosis are two different problems. Right now, it's only solved one of them. The paper is the coding-agent entry in the August 3 research roundup, benchmarked directly against AgentCoder rather than reported alone.
Each link below shares sources, entities, or timing with this story.
The Pragmatic Engineer published a deep read on August 25 of Inspect, the coding agent Ramp built instead of standardizing on Claude Code or Cursor. The numbers: Inspect authors 75% of Ramp's merged PRs, 90% of PRs in its own repository, passed 1 million total sessions in July...
The training-free method builds memories from historical traces summarizing reasoning patterns, key constraints and critical operations, then retrieves them as prefill-side scaffolds. Gains of 21.4, 28.0, 29.5 and 6.61 points on GSM8K, MATH, BBH and MMLU-Sci, with a 1.14-1.49x...
Numbers first, because they're the whole argument. On AppWorld's 168 tasks with DeepSeek-V3.2: ALTK-Evolve hit 89.3% goal completion at 263K tokens per task. ACE hit 80.4% at 634K (IBM Research on HF). Nine points better, 41% of the tokens. On the weaker gpt-oss-120b: 56.0% at...
arXiv 2608.04735 points out that monitorability evals overwhelmingly use *explicit* influence, where the prompt tells the model to hide a side task, and monitors catch 60-94% across seven frontier extended-thinking models. Swap in subtle contextual bias and detection drops 41-...
Rewriting an agent's reasoning to read as good-faith engineering while copying every command and output verbatim, so the exploit itself is unchanged, defeats a held-out CoT monitor almost completely. The sharper point for anyone running a monitor in production: headline accura...
What if the chain-of-thought isn't driving the answer? What if it's a post-hoc story the model tells itself? A new paper on arXiv titled "Therefore I Am. I Think" ran linear probes on reasoning model internals and found something uncomfortable. Tool-calling decisions are detec...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.