ProofAgent Index: a four-dimension readiness score arguing capability benchmarks do not predict production readiness

arXiv·medium signalXBlueskyLinkedInCopy link

The PAI combines Evaluation (observed behavior), Context (operating environment), Compliance (alignment with applicable rules), and Governance (whether the org can authorize, monitor, audit, and control the agent at runtime) into a single release-gate index, implemented in the open-source ProofAgent Harness. Validation across healthcare and finance shows PAI carries held-out readiness signal and separates higher-risk from lower-risk configurations, with three findings that cut against current practice: context engineering strongly changes reliability, capability improves behavior but does not determine readiness, and governance evidence degrades when averaged into a composite. The framing — reframing agent release from a faith-based to an auditable decision — lines up with the EU AI Act enforcement powers activating this week.