Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-16 · high
A study rewrote 120 ICLR papers into 4,200 versions and found evidence framing swayed scores more than the science, especially for papers in the middle.
Why now: The paper is newly reported as of August 16, 2026.
Researchers rewrote 120 anonymized ICLR 2026 submissions into 4,200 versions, then had five LLM reviewers score each one, per a paper posted to arXiv.
The result matters for any conference leaning on LLM reviewers, since wording can move a score more than the underlying result does.
Two LLM rewriters pushed six rhetorical dimensions, including evidence framing and novelty stance, in opposing directions on each paper while keeping the reported science fixed. Evidence framing and novelty stance moved scores the most. Scope framing followed as a weaker second factor.
The effect follows a pattern. Papers that started with low scores climbed. Papers that started high dropped. The swing showed up clearest in papers sitting in the middle of the original score range. Elaborate rewriting workflows didn't improve on simpler ones. Rewriting guided by what reviewers said didn't beat an unguided second pass at the same text.
A paper's phrasing moved its score almost as much as evidence framing did, with the findings themselves untouched. A review process built on more LLM reviewers doesn't correct for that bias. It amplifies whichever rhetorical dial a rewriter turns. That hits hardest on papers sitting in the middle of the pack, the ones most likely to be argued over in a real review cycle.
Each link below shares sources, entities, or timing with this story.
Simon Willison released LLM / Shared entity: LLM / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-07-27.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-19.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-18.
Simon Willison released LLM / Shared entity: LLM / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-07-31.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-19.
Simon Willison released LLM / Shared entity: LLM / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; pushes against this story (pushback).
Simon Willison released LLM / Shared entity: LLM / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-08-12.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-08-07.