GUI agents that score well on feasible tasks keep clicking through instructions that contradict the screen
CONFLICTGUI is a benchmark for conflict-aware termination covering instruction-internal conflicts and instruction-versus-GUI-context conflicts, built on the observation that real users issue infeasible instructions by ordinary mistake. Evaluation shows severe execution-biased overcompliance: the agents that do best on feasible tasks are the ones that blindly keep executing under conflicting instructions. The proposed CONFLICTGUARD is inference-time only, pairing a feasibility verification protocol that forces the agent to weigh instruction logic against GUI-side evidence with a conditional action modulation mechanism, and it raises average conflict-task success across five widely used agents while leaving normal GUI performance intact.
Source
↳ Follow the thread