Dispatch
AI Observatory: Anthropic's Economic Index filters out 48% of real Claude conversations
MIT Technology Review covers a Stanford/MIT effort (Anka Reuel, Shayne Longpre) that analyzed 24,521 donated conversations across 52 models from 2023–2025 and compared them against the vendors' own usage reports. Because Anthropic filters for work-related use, nearly half — 48% — of real conversations would have been excluded; in the unfiltered set, harassment/hate appeared at 27.5% versus Anthropic's reported 5.66%, sexual content at 16.7% versus 2.4%, and health/relationships at 44.2% versus 31.2%. Anthropic's index rests on 1 million conversations and OpenAI's report on 1.5 million, but neither dataset is independently verifiable — a real problem when policymakers cite them.
↳ Follow the thread