176-config study finds one context trick that wins

Researchers behind a paper posted September 17 built 176 harness configurations, held the execution loop fixed, and varied planning, action space, and context management independently across four models. The goal was isolating which harness decisions actually move coding-agent performance instead of crediting the model.

Context management came out on top under tight token budgets. The winning recipe was rule-based elision applied before LLM-based summarization, meaning the harness strips out content with fixed rules first and only hands the rest to a summarizer. Letting the agent recover elided text later added almost nothing to the score, which argues against the more complex recoverable-memory designs some harnesses build.

Planning split by model strength. For weaker models it functioned as accuracy scaffolding, catching mistakes a smaller model would otherwise make. For stronger models it only cut cost, since the model got to the right answer either way. That's a reason to stop treating planning modules as a universal quality lever and start asking which capability tier they're actually paying for.

The action-space result is narrower than it sounds. Bash-only interfaces beat predefined tool sets, but only for models already proficient with bash, per the study. It's a configuration rule tied to model choice, not a claim that tool abstractions are worse in general.

This is a separate paper from the Berkeley harness-tax dashboard that surfaced the day before, and it goes further: instead of scoring harnesses as whole systems, it isolates which piece inside the harness is doing the work. If you're building a coding agent, the actionable part isn't "add planning" or "add memory management" as a checkbox. It's picking rule-based elision over pure summarization, and checking whether your model is bash-fluent before deciding what action interface to expose.