Fetching from the wire…
01
02
03
04
05
06
07
08
Research2026-08-29 · source-backed
AgentJudgeBench, built from 3,808 instances stratified by difficulty, found judge alignment degrading monotonically with task difficulty, 1.5x faster with no ground truth available. On hard queries without ground truth, all six judges converged into that narrow band, pointing at a workflow-complexity ceiling rather than model capacity. Supplying reference answers made the stronger models worse through over-anchoring. Structured rubrics were the only intervention that helped, at up to 6.5 points; chain-of-thought and temperature changes did nothing. (arXiv)
Each link below shares sources, entities, or timing with this story.
Simon Willison released LLM / Shared entity: LLM / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; overlapping topics (chang, model).
LLM uses OpenAI / Shared entity: LLM / Shared topic / Earlier coverage
Linked by a graph relationship (LLM uses OpenAI); both cover LLM; overlapping topics (model, task).
Simon Willison released LLM / Shared entity: LLM / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; overlapping topics (answer, model).
Simon Willison released LLM / Shared entity: LLM / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-08-16.
LLM uses OpenAI / Shared entity: LLM / Earlier coverage / Tension
Linked by a graph relationship (LLM uses OpenAI); both cover LLM; earlier LLM coverage from 2026-07-27.
Simon Willison released LLM / Shared entity: LLM / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-19.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-18.
LLM uses OpenAI / Same source domain / Shared topic
Linked by a graph relationship (LLM uses OpenAI); reported by the same outlet (arxiv.org); overlapping topics (ceiling, model).