Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-07-23 · high
The paper finds the standard fail-to-pass test lets wrong patches pass, then fixes it with a 9.6-point resolution gain.
Why now: The paper posted to arXiv in July 2026, benchmarking CoHarden against the strongest existing test-and-fix co-generation approach on SWE-bench Verified.
Agent-written tests meant to verify bug fixes let broken patches pass as correct, according to a paper benchmarking repair systems on SWE-bench Verified.
That's the fail-to-pass check most automated repair systems use as a pass/fail gate. The test fails on buggy code, passes on the real fix, and counts as proof the bug is gone. Many of these tests turn out to be lax, per the paper. They reproduce the symptom but still pass patches that don't actually fix the underlying bug.
Worse. When a system generates the test and the fix in the same pass, the two can fail together in matching ways. The paper calls this error coupling: an in-trajectory check that says pass even though both the test and the patch are wrong.
The fix, called CoHarden, writes the test before the fix. It then repeatedly hardens both against mutation patches, alternate patches that pass the current test but shouldn't, until none survive. That gets to 78.9% correct fail-to-pass tests and 69.4% Resolved on SWE-bench Verified, 9.6 points above the strongest baseline that generates test and fix together.
Repair pipelines that score a bug fixed once the agent's own test passes are measuring the wrong thing. The paper's point is that this failure is silent by design: the test agrees with a wrong patch and nobody notices until the bug resurfaces.
Each link below shares sources, entities, or timing with this story.
Alibaba released Qwen3.6-27B on April 22. Dense architecture. Open weights. 77.2% on SWE-bench Verified, within 3.7 points of Claude Opus 4.6. On SkillsBench, it scores 48.2% versus its own 397B MoE predecessor's 30.0%. That's a 77% improvement with 14.8x fewer parameters. Let...
Allen Bargi's August 15 post hit 302 points arguing that AI collaboration rewards context-sharing, examples, and feedback over precise instruction (Hacker News). The pushback holds that the piece conflates management with leadership. mikeocool calls it "the most low effort ver...
Satya Nadella said companies routing everything through a single proprietary lab may not survive. His argument: you hand that lab your most sensitive business context, and the lab can turn it against you as a competitor. His prescription is an orchestration layer — keep the ha...
Willison launched datasette-apps (0.1a2) on June 18, hosting self-contained HTML+JS apps in a sandboxed iframe that run SQL against your data, read-only by default. He frames it as "Claude Artifacts reimagined for Datasette," artifacts backed by a JSON API to a relational data...
His conclusion is DuckDB matches or beats SQLite's safety for untrusted queries, but only with enable_external_access=false, lock_configuration=true, and a watchdog thread, since DuckDB lacks SQLite's opcode-based query timeouts. He ships a safe_duckdb.py helper and a Datasett...
OpenAI shipped GPT-5.5 on April 23, six weeks after 5.4. The capability jump is real: 82.7% on Terminal-Bench 2.0 vs Claude Opus 4.7's 69.4%. The Pro tier nearly doubles Opus 4.7 on FrontierMath Tier 4 at 39.6% vs 22.9%. It uses 40% fewer tokens on Codex tasks while matching 5...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.