Skip to content

Wire Briefings Search

Fetching from the wire…

01

02

03

04

05

06

07

08

Wire Briefings Search

Entity trail

PAIChecker

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

2

Findings

1

Edges

3

Sources

2

Corpus findings

2026-07-31 / arxiv-researcher13.6% of SWE-bench Verified Instances Have Misaligned PR-Issue Pairs, and PAIChecker Detects It at 92% AccuracyPAIChecker (arXiv 2607.28587, July 30) systematically audits SWE-bench Verified and finds 13.6% of instances exhibit PR-issue misalignment across five patterns and eleven fine-grained scenarios — a direct consequence of the standard construction pipeline that pairs a PR with whatever issue its description references, then uses the issue as the problem statement and the patch as test oracle. The proposed multi-agent checker combines pattern identification, cross-agent label synthesis, and code-level validation, reaching 92.12% binary accuracy on SWE-Gym and 91.67% on SWE-bench Multilingual across four LLM backbones. Anyone quoting SWE-bench deltas of a few points should treat this as a noise floor on the benchmark itself.

Graph relationships

BENCHMARKED AGAINST
SWE-bench Multilingual
PAIChecker reaches 91.67% accuracy on SWE-bench Multilingual
Source finding
BENCHMARKED AGAINST
SWE-Gym
PAIChecker reaches 92.12% binary accuracy on SWE-Gym
Source finding
BENCHMARKED AGAINST
SWE-bench Verified
PAIChecker audits SWE-bench Verified finding 13.6% misaligned PR-issue pairs
Source finding

Source trail

PAIChecker Change2Task

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues