A reward-design framework turns natural-language objectives into conflict-free reward functions in O(n log κ) preference queries

arXiv·low signalXBlueskyLinkedInCopy link

Di Yang Shi and W. Bradley Knox present a three-step process that lets non-experts convert natural-language task descriptions into linear reward functions: objective derivation that decomposes a task into fundamental objectives and measurable outcome variables, reward-term selection reformulated as a minimum-cost partial cover problem on causal DAGs solvable via maximum-flow, and weight fitting posed as a convex feasibility problem refined by iterative preference queries with separation-oracle efficiency of O(n log κ). The authors claim it is the first reward-design method maintaining a deterministically conflict-free feasible weight region. For teams doing RL-based agent tuning, the causal-DAG framing of which terms to include is the transferable idea.