UI-Voyager: Self-Evolving GUI Agent That Learns from Failed Trajectories

arXiv·medium signal

UI-Voyager presents a self-evolving mobile GUI agent that efficiently learns from failed interaction trajectories using multimodal LLMs. Unlike existing methods that discard failures, UI-Voyager implements ambiguous credit assignment to identify which actions in a failed trajectory were actually wrong, enabling targeted learning. The approach addresses a core challenge in agent development — converting failure experience into actionable improvement without human labeling.