The Boundaries of Automation: Human Participation Persists Because Some Targets Only Emerge Through the Interaction

arXiv 2607.21547·medium signal

Fourati, Schütze, Hüllermeier and Gurevych challenge the assumption that humans stay in the loop only because AI is not yet capable enough, identifying three durable grounds for persistent human participation: complementarity (humans supply capabilities AI lacks), normative/developmental (participation itself is valuable for agency or learning), and — the one they weight most heavily — target emergence, where the goal is not fully specified in advance but emerges through the interaction. Under target emergence, human participation is constitutive of the output rather than a means of improving execution. The practical implication is for evaluation design: benchmarks that assume a fixed target cannot measure systems operating in emergent-target domains.