Astra's hallucination numbers are the buried result: AA-Omniscience 92% to 51%, internal benchmark 12.2% to 4.2%

Artificial Analysis·high signalXBlueskyLinkedInCopy link

Artificial Analysis measured GPT-6 Astra's hallucination rate on AA-Omniscience at 51% at max effort against 92% for its predecessor, with accuracy up 4 points at the same time rather than traded away, and OpenAI's own system card reports an internal hallucination benchmark falling from 12.2% to 4.2%. An r/OpenAI thread at 275 upvotes complains this was buried in the system card while benchmark scores took the coverage. For anyone running unattended agent loops this matters more than the index score, because refusal-to-guess behavior is what determines whether a long chain survives.