Agents
Princeton Proposes 12-Metric Agent Reliability Framework: 90% Task Success Can Mask Unacceptable Autonomous Risk
Princeton's Narayanan and Kapoor published 'Towards a Science of AI Agent Reliability,' proposing 12 concrete metrics across four dimensions: consistency (same task same result), robustness (degraded conditions), predictability (calibrated uncertainty), and safety (bounded error severity). Their core finding: an agent succeeding on 90% of tasks but failing unpredictably on 10% may be a useful assistant but an unacceptable autonomous system. Fortune's March 24 coverage contextualizes this as the reliability framework the enterprise deployment gap has been waiting for — a vocabulary for what 'production-ready' actually means beyond average accuracy.
Source
↳ Follow the thread