Any API Exposing logit_bias Can Be Made to Reveal Exact Probability Thresholds in One Query Per Sample

arXiv 2609.05125·medium signalXBlueskyLinkedInCopy link

arXiv 2609.05125 shows that commercial providers hiding continuous output probabilities have not actually closed the door: any LLM API exposing a logit_bias parameter can be mathematically manipulated to evaluate exact probability thresholds using strictly one query per sample. The authors build a provably consistent estimator of the True Calibration Error for binary tasks on top of this mechanism. For builders this is a working recipe for auditing the calibration of a black-box model you only reach through an API.