Fetching from the wire…
01
02
03
04
05
06
07
08
Research2026-08-13 · source-backed
arXiv 2608.11434 built MobileJudgeBench from 931 human-annotated trajectories across 6 benchmarks, 4 agent models, and 68 apps, then evaluated 6 judge methods adapted from SPA-Bench, A3, AndroidArena, and AgentRewardBench. A baseline judge fed sampled screenshots is competitive with and often exceeds the elaborate pipelines. The LLM backbone, not the method, is decisive. If you're building LLM-as-judge infrastructure, spend your budget on the model, not the scaffold.
Each link below shares sources, entities, or timing with this story.
Simon Willison released LLM / Shared entity: Bench / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover Bench; overlapping topics (agent, benchmark, model).
Simon Willison released LLM / Shared entity: LLM / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; overlapping topics (agent, model).
Simon Willison released LLM / Same source domain / Shared topic / Tension
Linked by a graph relationship (Simon Willison released LLM); reported by the same outlet (arxiv.org); overlapping topics (agent, model).
Simon Willison released LLM / Shared entity: LLM / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; overlapping topics (agent, model).
Simon Willison released LLM / Shared entity: LLM / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-07-27.
Simon Willison released LLM / Shared entity: Bench / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover Bench; earlier Bench coverage from 2026-04-25.
Simon Willison released LLM / Shared entity: LLM / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-19.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-18.