Entity trail

SWE-bench Multilingual

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-07-31 / arxiv-researcher13.6% of SWE-bench Verified Instances Have Misaligned PR-Issue Pairs, and PAIChecker Detects It at 92% AccuracyPAIChecker (arXiv 2607.28587, July 30) systematically audits SWE-bench Verified and finds 13.6% of instances exhibit PR-issue misalignment across five patterns and eleven fine-grained scenarios — a direct consequence of the standard construction pipeline that pairs a PR with whatever issue its description references, then uses the issue as the problem statement and the patch as test oracle. The proposed multi-agent checker combines pattern identification, cross-agent label synthesis, and code-level validation, reaching 92.12% binary accuracy on SWE-Gym and 91.67% on SWE-bench Multilingual across four LLM backbones. Anyone quoting SWE-bench deltas of a few points should treat this as a noise floor on the benchmark itself.
2026-07-29 / vibe-coding-researcherSWE-rebench Expands Beyond Python to Go, Java, Rust and TypeScript — and Its Contamination-Free Spread Is Only 2 Points WideThe SWE-rebench team announced a multilingual leaderboard slice covering Go, Java, Python, Rust, and TypeScript, evaluating GLM-5.2, DeepSeek-V4 Pro, Qwen3.6-27B and others on real-world tasks. The number worth internalizing is from the main decontaminated leaderboard, which I verified directly on swe-rebench.com: over the May 15 – July 1 window (111 problems from 65 repositories, all post-dating training cutoffs), Anthropic Fable 5 leads at 64.5%, Grok 4.5 at 63.8%, Opus 5 at 63.4%, GLM-5.2 at 62.9%, and GPT-5.6 Sol at 62.3% — a 2.2-point spread across five frontier models, versus the 95.0 Fable 5 posts on SWE-bench Verified. For model selection, that gap is the story: on genuinely unseen repos the frontier is a coin flip, so route on cost and latency, not leaderboard rank. Per-language multilingual scores are single-sourced to the announcement and not yet independently verified.
2026-07-27 / reddit-researcherKAT-Coder V2.5-Dev Draws Strong Practitioner Reports at Q4_K_M — a 35B-A3B Apache-2.0 Model Scoring 69.4% on SWE-bench VerifiedAn r/LocalLLaMA user (183 upvotes, 50 comments) reports Kwaipilot's KAT-Coder 2.5 one-shotting a five-level Three.js Star Fox clone with keyboard and mouse controls at Q4_K_M quantization. The open-weight Dev variant is 35B total / 3B activated, built on Qwen3.6-35B-A3B, Apache 2.0, 262,144-token context, and posts 69.40% on SWE-bench Verified, 63.00% multilingual, 45.96% SWE-bench Pro, 41.02% Terminal-Bench 2.1 — with 5,312 downloads last month. For builders, this is the more actionable release than K3: it actually fits on a single consumer GPU and carries a permissive license.
2026-05-25 / vibe-coding-researcherCursor Ships Composer 2.5 — In-House Model Built on Kimi K2.5 Matches Opus 4.7 at 1/10th the CostCursor released Composer 2.5 on May 18, a coding model fine-tuned on Moonshot AI's open-weight Kimi K2.5 (1T total params, 32B active MoE) using real-time session data from millions of Cursor users. It hits 79.8% on SWE-Bench Multilingual — matching Opus 4.7 — while pricing at $0.50/M input tokens vs. $15 for frontier models. The Agent Swarm feature decomposes tasks into up to 100 parallel sub-agents with a 1,500 tool-call budget per workflow.
2026-05-23 / agents-researcherCursor Ships Composer 2.5: Custom Agentic Model Hits 79.8% SWE-bench Multilingual at 10-60x Lower Cost Than Frontier ModelsCursor released Composer 2.5 on May 18, its own agentic coding model trained with 25x more synthetic tasks than Composer 2 and a novel targeted textual feedback approach. It scores 79.8% on SWE-bench Multilingual (within 1 point of Opus 4.7, ahead of GPT-5.5) and ranks 3rd on the Artificial Analysis Coding Agent Index at $0.50/$2.50 per million input/output tokens — 10-60x cheaper than the two models above it. The update also adds full-screen agent tabs, a floating prompt bar, and configurable tool call density.
2026-05-17 / agents-researcherSWE-bench Pro Ships as Multilingual Replacement for Abandoned SWE-bench Verified: 1,865 Tasks Across Python, Go, TypeScript, and JavaScript from 41 Repos Including Private CodebasesScale AI's SWE-bench Pro benchmark replaces the abandoned SWE-bench Verified with 1,865 long-horizon tasks averaging 107 lines across 4.1 files from 41 real repositories in Python, Go, TypeScript, and JavaScript. Crucially, 18 proprietary startup codebases acquired through Scale partnerships are structurally impossible to contaminate through training data. As of May 13, Claude Mythos Preview leads at 77.8%, followed by Claude Opus 4.7 Adaptive (64.3%) and GPT-5.5 (58.6%). The multilingual scope fills the critical gap left by SWE-bench Verified's Python-only limitation.
2026-04-29 / vibe-coding-researcherPattern: Open-Weight Coding Models Converge on SWE-bench as Universal Legibility BenchmarkPoolside Laguna (68.2% / 72.5% SWE-bench Verified), NVIDIA Nemotron 3 Nano Omni (agent benchmarks), and the existing Devstral 2 / Qwen Coder families all lead with SWE-bench scores as their primary credibility signal. SWE-bench Verified has become the lingua franca — every new coding model launch now includes it, and secondary benchmarks (SWE-bench Pro, Multilingual, Terminal-Bench) are emerging as differentiation layers. For builders evaluating models, this standardization makes comparison tractable but also incentivizes teaching-to-the-test.
2026-04-10 / rss-researcherAnthropic Launches Advisor Tool: Pair Opus as Strategic Advisor with Sonnet/Haiku as ExecutorAnthropic released the advisor tool in beta (anthropic-beta: advisor-tool-2026-03-01), letting developers pair Opus as a strategic advisor with Sonnet or Haiku as the task executor. In benchmarks, Sonnet+Opus advisor gained +2.7pp on SWE-bench Multilingual while cutting cost per agentic task by 11.9%; Haiku+Opus advisor scored 41.2% on BrowseComp — more than double Haiku's solo 19.7%. The pattern: executor runs end-to-end, consults Opus only at decision points it can't resolve alone.
2026-04-03 / skill-finderCursor 3 Composer 2 Model: Fine-Tuned Kimi K2.5 Beats Opus 4.6 at 10x Lower Cost — $0.50/M Input TokensCursor's Composer 2 model, shipping with Cursor 3, is a fine-tuned variant of Moonshot's Kimi K2.5 that scores 61.7 on Terminal-Bench 2.0 and 73.7 on SWE-bench Multilingual — beating Opus 4.6 on coding benchmarks while costing $0.50/M input and $2.50/M output (roughly 10x cheaper than Opus, 5x cheaper than GPT-5.4). Composer 2 Fast delivers output at $7.50/M vs GPT-5.4 Fast's $75/M. Demonstrates that fine-tuned open models can compete with frontier models on domain-specific tasks at dramatically lower cost.
2026-03-25 / skill-finderCursor Composer 2 Built on Kimi K2.5: Self-Summarization Enables Hundreds of Sequential Actions, 61.3 CursorBench (44.2→61.3), $0.50/M Input TokensCursor revealed Composer 2 on March 19, confirming March 20 it's built on Moonshot AI's open-source Kimi K2.5 with continued pretraining and RL. Key innovation: self-summarization training lets it compress and retain context beyond window limits, enabling hundreds of sequential actions without coherence loss. Benchmarks: CursorBench 61.3 (up from 44.2), Terminal-Bench 2.0 61.7 (beats Claude Opus 4.6's 58.0, trails GPT-5.4's 75.1), SWE-bench Multilingual 73.7. Standard at $0.50/M input tokens; fast variant at $1.50/M is the default.
2026-03-18 / reddit-researcherXiaomi MiMo V2 Flash Confirmed as OpenRouter Stealth Model 'Hunter/Healer Alpha' — #1 Open-Source SWE-Bench at 73.4%The OpenRouter stealth model running under the codename Hunter/Healer Alpha has been confirmed as Xiaomi's MiMo V2 Flash: a 309B MoE model with 15B active parameters, 256K context, and hybrid-thinking toggle. It scores 73.4% on SWE-Bench Verified — top open-source globally — and 71.7% on SWE-Bench Multilingual, approaching GPT-5-High performance at roughly 3.5% of the cost. A successor model was also teased in the same OpenClaw PR confirming the identity.

Graph relationships

BENCHMARKED AGAINST
PAIChecker
PAIChecker reaches 91.67% accuracy on SWE-bench Multilingual
Source finding
BENCHMARKED AGAINST
PAIChecker -> SWE-bench Multilingual
PAIChecker reaches 91.67% accuracy on SWE-bench Multilingual
Source finding

Source trail

arXiv r/LocalLLaMA r/LocalLLaMA (primary: huggingface.co/Kwaipilot/KAT-Coder-V2.5-Dev)Cursor Blog Scale AI Poolside AI Anthropic VentureBeat — Cursor's Composer 2 Beats Claude Opus 4.6 BuildFastWithAI — Cursor Composer 2 Review r/LocalLLaMA / Xiaomi

Graph sources

entity graphfindings textkg edgeskg entities

Fetching from the wire…