Gated Hindsight Distillation trains GUI agents on the screenshot that comes AFTER the action

arXiv·low signalXBlueskyLinkedInCopy link

Standard offline GUI-agent training decomposes trajectories into prefix-action pairs and discards the next observation — which is often the only place the rationale lives (nothing reveals that Soft Wrap is under Edit or View until the menu opens), so imitation gives the model little chance of ever sampling the correct reasoning. GHD (arXiv 2608.06065, submitted 2026-08-06) uses the next screenshot as privileged information during training only: a parameter-sharing teacher that sees it re-scores the student's on-policy responses, and distillation fires only when the student fails and the hindsight-conditioned teacher recovers the demonstrated action. It improves task success over GRPO on AndroidWorld and AndroidLab across two vision-language models, with code and checkpoints promised.