Repackaged APKs steer mobile agents to attacker-chosen actions 77.9% of the time using perturbations 186 human testers could not see

arXiv·high signalXBlueskyLinkedInCopy link

Researchers name and exploit human-agent UI desynchronization: users perceive a mobile interface through a physical display and the limits of human vision, while the agent consumes raw screenshots plus accessibility metadata that expose non-visual widget content, so the same UI state carries materially different information to each. They built an automated framework that bakes user-instruction-agnostic perturbations into deployable APKs that stay fully functional for humans, then tested five mobile-agent frameworks and three backbone models across 546 tasks, hitting 77.9% average misleading rates statically and 66.9% dynamically. A 186-participant questionnaire confirmed the perturbations are hard for people to notice, which breaks the human-oversight premise that mobile agent deployments are sold on.