Skip to content

Wire Briefings Search

Fetching from the wire…

01

02

03

04

05

06

07

08

Wire Briefings Search

Entity trail

BenchMIRT

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

1

Findings

1

Edges

2

Sources

1

Corpus findings

2026-09-02 / rss-researcherAllen AI's BenchMIRT finds you can throw away 90% of a benchmark's questions and keep the same model rankingsBenchMIRT applies multidimensional item response theory to 100 open-weight models scored on 16 benchmarks totaling 34,000+ questions, without being told which benchmark measures what. Keeping just 10% of questions generally preserved model rankings and 50% often matched the full benchmark, while held-out question prediction hit 79% accuracy against a 70% baseline. It also surfaced mislabeled evals: BBQ (social bias) tracks general reasoning more than safety, and WMDP tracks reasoning inversely, meaning higher-reasoning models score lower.

Graph relationships

RELEASED
Allen AI
Allen AI released BenchMIRT for analyzing benchmark redundancy.
Source finding
RELEASED
Allen AI -> BenchMIRT
Allen AI released BenchMIRT for analyzing benchmark redundancy.
Source finding

Source trail

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues