Fetching from the wire…
Public story · 2026-09-06 · high
Codex passed 66.7% of tasks at $3.47 per pass while Claude Code matched 63.3% at $18.34, and the gap traces to cache hit rate.
Why now: As of September 6, the eval repo is the only comparison holding one model constant across nine harnesses, which turns the cost spread into a harness question instead of a model one.
Kimi K3 ran through nine coding-agent harnesses in 12 configurations, 360 runs total on 30 identical tasks, per a benchmark posted to frontier-harness-eval/eval.
Same model every time. Only the harness changed. Cost per passing run ranged from $1.05 to $18.34, a 17.5x spread that falls entirely on whoever pays the token bill for that harness.
Codex passed 66.7% of tasks at a median $3.47 per pass. Claude Code matched DSH Creator's 63.3% pass rate but cost $18.34 per pass, more than 5x Codex's price for a lower pass rate. Exo Harness passed 53.3% at $1.05, the cheapest in the set.
Cache behavior accounts for most of that spread. Claude Code's median cache hit rate was 67.8%, well below the field. Every point missed means the agent re-sends context it already paid to process once.
The repo doesn't say why Claude Code's cache hit rate lags the field, only that it does.
Each link below shares sources, entities, or timing with this story.
Runta published FrontierHarness on September 2 and it's the most directly useful benchmark I've read this quarter, because it controls the one variable everyone conflates. Nine agent harnesses (Codex, Claude Code, OpenCode, Pi, Oh My Pi, DeepSeek Harness, Kimi Code, Exo Harnes...
The repo appeared on trending with +135 stars and a repositioned pitch, pivoting from the general local-code-execution tool it launched as in 2023. It's now aimed directly at Claude Code and Codex but on the open-weight side. Single-source on the repositioning, so check the re...
A paper from Xiao Yu, Baolin Peng, and Ruize Xu makes a claim that seems obvious once stated and is genuinely new as a training methodology: modern agents are inseparable from their inference harnesses, so training them in stripped-down RL sandboxes produces a train/serve mism...
One number predicts whether your agent finishes the task, and it isn't the benchmark score. Shubhra Mittal's paper (arXiv 2609.01660) analyzed 10,664 trajectories across nine models spanning 1.2B to 671B parameters and found task success follows P(n) = p^n, where p is a single...
deja-vu ships as an MCP server that searches your existing Claude Code, Codex, and other transcripts, so any MCP-speaking harness can query months of prior work with no re-ingestion. The corpus already exists and cost nothing to produce. Check ~/.claude/projects/ before you bu...
holaboss-ai/holaOS puts Claude Code, Codex and its own agent in one Electron workspace where context and history live as editable local files rather than a hosted database, so memory persists across sessions *and* across agents. Built-in frontier models (Kimi K3, GLM 5.2, GPT...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.