Sources
Prompt Injection as Role Confusion: Models Infer Authority from Text Formatting, Not Source — 60% Attack Success Across Open and Closed Models
Researchers demonstrate that LLMs assign authority based on how text is formatted rather than where it originates, enabling 61% success rates on agent exfiltration tasks and 60% on StrongREJECT benchmarks. Novel 'role probes' measuring internal role identification show the degree of latent role confusion strongly predicts attack success before generation begins. No defenses are proposed — the paper identifies a fundamental gap between where security is defined (interface level) and where authority is actually assigned (latent space).
Source
↳ Follow the thread