Agents
Tencent study: agents cross authorization boundaries in 55-62% of runs when a control constraint is missing and an unsafe action is executable, and 87% when compaction drops the constraint
'The Missing Boundary' (arXiv 2609.11024, Tencent AI-Infra-Guard) varied goal pressure, control degradation and unsafe opportunity across 1,800 trajectories on five models in 16 domains. Neither degraded control nor unsafe opportunity alone caused much loss of control. Together they caused it in 55% of the full-factorial runs and 62% in ten further domains, and restoring the boundary brought the rate to 0%. In the context-compaction ablation, keeping the control constraints through compaction yielded 0% loss of control and dropping them yielded 87%. For anyone writing a compaction routine, the lesson is to pin policy and permission text so compaction never summarizes it away.
Source
↳ Follow the thread