OpenForgeRL: the harness is the environment, and training outside it is a mismatch

A paper from Xiao Yu, Baolin Peng, and Ruize Xu makes a claim that seems obvious once stated and is genuinely new as a training methodology: modern agents are inseparable from their inference harnesses, so training them in stripped-down RL sandboxes produces a train/serve mismatch.

OpenForgeRL trains policies directly against the harness. Claude Code, Codex, OpenClaw, in arbitrary environments. The multi-turn reasoning structure, the tool routing, the environment access, all of it is treated as part of the learned system rather than as scaffolding you bolt on after.

Think about what this implies. If an agent is trained in a clean gym where tool calls return instantly and deterministically, and then deployed into a harness where a tool call might time out, return truncated output, or hit a permission dialog, the policy has never seen the distribution it operates in. The gym taught it a world that doesn't exist.

This is the day's through-line, and it connects to something more directly measurable. A longitudinal study evaluated 35 sequential Qwen Code CLI releases against 50 stratified SWE-bench Verified tasks with the underlying LLM held constant. Quality fluctuated. They traced the fluctuations to individual pull requests and architectural components.

Same model. Different harness version. Different results.

Major open-source harnesses ship more than two releases per day. So when you notice your agent got worse this week, the reflex is to blame the model, and that reflex is frequently wrong. You updated the harness. Or the harness updated itself.

Three actions, all cheap:

Pin your harness version in CI. Not "latest." A specific version, bumped deliberately.

Keep a small stable task set, ten to twenty tasks, that you re-run on every harness bump. Not a real benchmark. Just enough signal to catch a regression.

Diff harness releases before you diff models. Read the changelog first.

I'll flag the tension: ECC exists precisely because people want to swap harnesses freely, and it adds skills, instincts, memory and security as a portable layer across Claude Code, Codex, Opencode and Cursor. The "instincts" primitive is new to me, a layer for reflexive behaviors distinct from invokable procedures. But if the harness is genuinely part of the learned system, then perfect portability across harnesses is a harder problem than a config translation layer. Both things are happening at once and I don't think they've been reconciled yet.