GraphRAG Detection Plans Survive 100% IOC Rotation Where Naive RAG Drops to 29%

arXiv 2608.13050·high signalXBlueskyLinkedInCopy link

This study feeds the same CTI report, same generation instructions and same LLM to Microsoft GraphRAG versus a standard vector-similarity RAG, varying only the retrieval step, then rotates every IP, domain and file hash in the report to test durability. In an APT28 case study the GraphRAG detection plan kept firing at 100% of its detections after rotation while the Naive RAG plan kept only 29%; the pattern replicates across nine real CTI reports from four vendors, with GraphRAG plans consistently reaching higher tiers of the Pyramid of Pain. The authors also note prompt wording matters almost as much as the retrieval backend — a useful caveat for anyone attributing gains to graph retrieval alone.