LLM Preference Judgments Are Not Self-Consistent, Which Breaks the Utility-Estimation Pipeline Built On Them

arXiv 2608.17644·medium signalXBlueskyLinkedInCopy link

A growing pattern has agents interpret natural-language preferences by asking an LLM for numeric judgments, such as willingness to pay, then fitting a utility function and choosing actions from it. arXiv 2608.17644 (2026-08-18) tests the assumption that pipeline requires, that a single utility function can reproduce the judgments, by measuring whether stated willingness-to-pay differences between two items match the stated indifference payment. They do not. That is a direct warning for anyone building preference-elicitation or personalization layers on top of LLM-scored comparisons.