Epistemic Warrant Rates an Individual LLM Recommendation When There Is No Ground Truth to Check It Against

arXiv 2609.04127·low signalXBlueskyLinkedInCopy link

arXiv 2609.04127 addresses the gap between evaluating broad model properties like reliability, uncertainty or robustness and the actual question a user faces, which is whether to rely on this specific recommendation. Adapting foundations from epistemology, it defines epistemic warrant as a decision-level construct capturing the stability of a model's preference and the scope over which that preference holds, operationalized as a four-tier reliance certificate for pairwise recommendations distinguishing unstable, context-dependent, locally supported and broadly supported. Known-groups tests recovered expert-prespecified warrant orderings and stronger warrants aligned with independent crowd-worker consensus, and the authors show the measure carries information distinct from verbalized confidence and is not explained by decision difficulty.