CTI-REALM: Benchmark to Evaluate Agent Performance on Security Detection Rule Generation Capabilities

arXiv 2603.13517·high signal

Microsoft Security AI releases CTI-REALM, a benchmark replicating the security analyst workflow: agents read threat intelligence reports, execute queries, parse schema structures, and build detection rules across Linux, cloud, and Azure Kubernetes attack scenarios. Two sub-benchmarks (CTI-REALM-25 and CTI-REALM-50) scale from atomic to complex multi-step intrusion scenarios. Evaluation of 16 frontier models places Claude Opus 4.6 High at the top (reward 0.637), ahead of Claude Opus 4.5 (0.624) and GPT-5.