Multi-Environment Machine Teaching Produces Reward Functions That Survive Context Shift

arXiv·medium signal

Inverse reinforcement learning infers objectives from human feedback, but existing analyses of optimal teaching assume a single environment and demonstration-only feedback, which produces reward functions overfitted to that environment. This paper (arXiv 2607.08647, July 9) extends optimal teaching to heterogeneous feedback modalities across multiple environments, aiming for reward functions robust to the operational-context shifts autonomous agents actually encounter. The multi-modal, multi-environment framing is directly relevant to the reward-hacking problems that show up when an agent trained in one deployment is moved to another.