XConf estimates agent confidence from its own past episodes and beats 10-sample self-consistency at a tenth the cost

arXiv / HuggingFace Daily Papers·medium signalXBlueskyLinkedInCopy link

arXiv 2609.17708, submitted 2026-09-15 by Zhang, Zhu, Li, Chen, Kumaran and Collier, scores confidence using a model's recorded history rather than the current inference alone. It recalls similar past episodes at comparable stated confidence to extract real success rates, then has the model name its recurring failure patterns and revise. Across nine benchmarks and four models from three families it matched or beat ten-sample self-consistency on AUROC in 23 of 24 comparisons with roughly a tenth the ECE and a tenth the generation cost, and selective prediction raised delivered agent success by up to 8.7 points.