AgentOPSD: Critic-Free Turn-Level Credit Assignment via Bayesian Belief Updates Hits 89.1% on ALFWorld With a 7B Model

arXiv / HuggingFace Daily Papers (57 upvotes)·medium signalXBlueskyLinkedInCopy link

Submitted Aug 6 (arXiv 2608.05987), AgentOPSD attacks the long-horizon credit assignment problem by recursively aggregating token-level signals into turn-level evidence through Bayesian belief state updates — requiring neither an extra critic network nor additional rollouts, so it drops into standard policy optimization. On ALFWorld with Qwen2.5-7B it reaches 89.1% success, beating GRPO and self-distillation baselines, with gains confirmed by ablation to come from the turn-level aggregation and history-dependent recursive updates rather than the base model. For anyone training agents on multi-step tool tasks, the appeal is cost: turn-level reward shaping without the rollout tax.