Harness-RL routes action and argument gradients into separate parameter subspaces to train the central agent of a multi-agent harness

arXiv·medium signalXBlueskyLinkedInCopy link

Training a coordinator policy hits two problems: the action label is a low-cardinality decision while its arguments are a high-dimensional conditional sequence, so a shared sequence-level signal produces conflicting gradients, and dynamic scheduling with branches, parallel calls and rewritten contexts cannot be flattened into one token sequence. Harness-RL captures Interface Call Records into per-session prefix trees and uses Conflict-Aware Policy Optimization, identifying parameter partitions by forward activation and routing each gradient accordingly. Across seven multi-hop QA and agentic retrieval benchmarks it reaches average F1 of 42.93 with Qwen2.5-1.5B and 47.79 with 3B, with ablations favoring central-only over joint optimization. Code is public.