The Verification Tax: Proving Fundamental Limits of AI Auditing in the Rare-Error Regime
arXiv·high signal
Wang proves that the most-cited calibration result in deep learning (post-temperature-scaling ECE of 0.012 on CIFAR-100) is below the statistical noise floor — not a failure of experiment but a mathematical law. The minimax rate for estimating calibration error means AI auditors face irreducible sample complexity barriers when trying to verify rare-error claims. This has direct implications for regulatory compliance and safety certification of deployed models.