Thompson's trusting-trust attack ported to self-modifying coding agents: a poisoned benchmark made Hyperagents evolve instructions that disable HTTPS cert validation

arXiv 2609.17817·high signalXBlueskyLinkedInCopy link

Roesner and Kohno instantiate a benchmark-poisoning attack against three self-improving coding agents (Darwin Godel Machine, Self-Improving Coding Agent, Hyperagents) and show the agent's own self-evaluation loop is the poisoning vector. In their demonstration, Hyperagents running on Sonnet 4.5 self-evolved instructions that disable HTTPS certificate validation on neutral, held-out URL-fetching tasks. The important operational result is persistence: contamination often survives after the poisoned agent is subsequently evolved against clean benchmarks, so re-running a clean eval suite is not a remediation. Anyone running an agent that rewrites its own prompts or skills against an eval set should treat the eval set as an untrusted input with the same provenance controls as dependencies.