Fetching from the wire…
01
02
03
04
05
06
07
08
Research2026-08-11 · source-backed
arXiv 2608.09567 pre-registered a screen of 104 papers from 2023–2026; only 59 (56.7%) have a publicly reachable artifact. Executing an 18-paper sample plus all 102 cases of an anchor benchmark, 58/102 anchor cases carry a script-internal CVE ID that diverges from the declared directory CVE, and only 10/18 artifacts complete their declared workflow at first run. The killer: artifact-embedded oracles show 60% sensitivity and 45% specificity, meaning 20 of 30 patched-counterfactual audits still produce the claimed signal on the patched build. A trigger on a vulnerable build is not evidence of CVE-specific reproduction.
Each link below shares sources, entities, or timing with this story.
Simon Willison released LLM / Shared entity: CVE / Same source domain / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover CVE; reported by the same outlet (arxiv.org).
Simon Willison released LLM / Shared entity: LLM / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-07-27.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-19.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-18.
Simon Willison released LLM / Shared entity: CVE / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover CVE; earlier CVE coverage from 2026-08-08.
Simon Willison released LLM / Shared entity: LLM / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-07-31.
Simon Willison released LLM / Shared entity: CVE / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover CVE; earlier CVE coverage from 2026-07-31.
Simon Willison released LLM / Shared entity: LLM / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-19.