Semantic similarity misses the model diversity that predicts correlated failure.

Multi-model systems treat different models as independent components even when failures stay correlated, and existing diversity metrics only capture differences in output meaning. The authors measure generative-process diversity via Normalised Compression Distance between raw outputs residualised against a permutation control. Across 38 models it identifies population structure semantic similarity misses and predicts chance-corrected correlated failure across ten disjoint benchmark families, cross-benchmark partial rank association -0.216 (95% interval -0.309 to -0.122), negative on all ten. If your fallback strategy is "route to a different model," this is how you'd check whether that helps. arXiv 2609.03422