Perplexity can't detect RAG poisoning — poisoned answers score *lower* perplexity; watch attention entropy collapse instead

arXiv 2608.06947·medium signalXBlueskyLinkedInCopy link

D-SCAN (SIGIR 2026) reports that perplexity-based RAG poisoning detection is actively misleading because adversarial generations often have lower perplexity than benign ones, so the standard guardrail returns high confidence on compromised output. Their signal is document-level attention dynamics: during a poisoned generation, attention concentrates on the injected document and entropy drops, versus the dispersed attention of a legitimate answer. It is a lightweight runtime check with code published at github.com/yingtaoren/D-Scan, and it applies to any self-hosted RAG stack where you can read attention — a real detection path for teams who assumed retrieval hygiene was enough.