Allen AI's BenchMIRT says you can throw away 90% of a benchmark and keep the rankings.

Multidimensional item response theory applied to 100 open-weight models on 16 benchmarks totaling 34,000+ questions, with no labels about what measures what. Keeping 10% generally preserved model rankings; 50% often matched full (Hugging Face). It also surfaced mislabeled evals: BBQ tracks general reasoning more than social bias, and WMDP tracks reasoning inversely, so higher-reasoning models score lower.