Entity trail

Terminal-Bench 3.0

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Showing the first 40 findings. More graph evidence exists in the corpus.

Corpus findings

2026-08-24 / reddit-researcherQwen 3.8 27B Reverse-Engineered Firmware From an Early-2000s ARM Point-of-Sale Terminal That Opus 4 Could NotA practitioner who previously benchmarked Qwen 3.8 against 3.6 on an HTML Galaga port deliberately moved to a task with no plausible training-data contamination: firmware and software preservation and emulation for an early 2000s ARM-based POS system. Running UD-Q8_K_XL on two RTX 3090s under llama-server with Opencode, the local model completed work he says Opus 4 could not, and the post got picked up by XDA Developers the same day. A commenter with 80 upvotes independently reported running an abliterated Qwen 3.8 alongside DeepSeek V4 Pro and headless Ghidra for binary analysis, which is a second data point that this model is unusually strong at non-x86 reverse engineering.
2026-08-21 / rss-researcherOrnith-1.5 ships 397B, 35B and 9B open weights under MIT and lands within a point of Claude Opus 4.8 on Terminal-BenchOrnith released Ornith-1.5 on 19 August 2026 as three MIT-licensed models: a 397B MoE, a 35B MoE, and a 9B dense with a quantized Mobile build that runs on iPhone and Android. The 397B scores 86.1 on Terminal-Bench 2.1 (Terminus-2) against Claude Opus 4.8's 85.0, 86.0 on SWE-bench Verified, and 92.8 on GPQA Diamond, but trails badly on the harder agentic rows (13.5 vs 21.1 on Frontier-Bench v0.1, 59.5 vs 69.7 on NL2Repo). All numbers are vendor-reported and averaged over five runs; weights for every size are already downloadable, unlike GLM-5.3 which is still API-only.
2026-08-20 / reddit-researcherOrnith-1.5 Ships 9B, 35B-A3B and 397B Open Weights Claiming Claude Opus 4.8 Parity, With 86.1 on Terminal-Bench 2.1The Ornith team published Ornith-1.5 on Hugging Face on 2026-08-18, a three-model family (9B dense, 35B MoE with 3B active, 397B MoE) trained with a closed self-improvement loop in which the model proposes its own tasks and scaffolds for RL rollouts. The 397B claims 86.1 on Terminal-Bench 2.1, 86 on SWE-Bench Verified, 56 on DeepSWE, 92.8 on GPQA Diamond and 44.6 on HLE, which the team frames as comparable to Claude Opus 4.8. A top commenter's side-by-side table shows the 35B-A3B losing to Qwen3.8-27B on Terminal-Bench (68.5 vs 73.0) and DeepSWE (22.0 vs 42.2) while winning NL2Repo (46.2 vs 42.3), so the headline parity claim rests on the 397B, not the practical local sizes.
2026-08-20 / github-pulse-researcherAutoprompt Skill Publishes a Measured Terminal-Bench 2.1 Run: OpenCode Goes From 60/89 to 73/89 SolvesSpielewoy/autoprompt-skill, created 2026-08-17 and already at 407 stars with three releases in three days (v1.0.0 on 08-18 through v1.0.2 on 08-19), is a coding-agent skill claiming 45% fewer failures. The README publishes the actual run rather than a marketing number: OpenCode 1.18.7 solved 60 of 89 tasks (67.42%) and OpenCode plus Autoprompt solved 73 of 89 (82.02%), a gain of 14.61 points and a drop from 29 failures to 16. Unusually, the repo also states the cost honestly, roughly 3x the time and 2x the tokens, and flags that those trade-off figures are planning estimates from user reports rather than retained logs.
2026-08-20 / hn-researcherOrnith-1.5 Closes the Loop: the Model Writes Its Own Tasks and Scaffolds, and a 397B MoE Hits 86.1 on Terminal-Bench 2.1Ornith released 1.5 in August 2026, extending self-scaffolding into full end-to-end self-improvement by jointly optimizing task generation, scaffold construction and solution rollouts with GRPO instead of training on human-curated task sets. The 397B MoE scores 86.1 on Terminal-Bench 2.1 and 56.0 on DeepSWE, which the writeup puts level with Claude Opus 4.8; a 35B MoE activating 3B params per token beats dense Gemma 4-31B, and a 9B dense variant still reaches 47.0 on Terminal-Bench 2.1 on-device. Built on Qwen3.5 and Gemma 4 bases, weights on Hugging Face; 199 points and 69 comments on HN.
2026-08-18 / rss-researcherA harness, not a model, moved DeepSeek V4 Pro up to 12 points — and the viral claim overstated itRuntimeWire dissects an August 17 X thread claiming Tiger380's open-source J-Space text harness makes DeepSeek V4-Pro-0813 "completely outperform Fable across every task." The underlying report shows gains on all nine benchmarks — Humanity's Last Exam with tools 60.0→67.7, NL2Repo 61.5→73.4, DeepSWE 62.7→72.0, Terminal Bench 2.1 87.9→90.1, CyberGym 83.3→86.8, Toolathlon-Verified 74.1→79.5 — but Fable 5 still leads HLE-without-tools at 53.3 vs 48.0 and GLM-5.3 leads AutomationBench 48.2 vs 38.2. Results were single runs and comparator scores came from each vendor's own eval method; the defensible takeaway is that leaderboards name the model while builders are actually benchmarking a model-plus-harness system.
2026-08-16 / saas-disruption-researcherCROSS-CATEGORY: Inference Pricing Bifurcated in 72 Hours — DeepSeek Went Metered-and-Up While Z.ai Went Flat-Rate Subscription at a Tenth of US RatesZ.ai shipped GLM-5.3 on Aug 14 — same 743B base as GLM-5.2, moved by post-training alone (Terminal-Bench 3.0 from 4.6 to 28.3, DeepSWE v1.1 from 46.2 to 66.9) — and priced it as GLM Coding Plan subscriptions at $18/$72/$160 per month, roughly one-tenth of US frontier rates, with the per-token API rate still unpublished and open weights staged behind a safety review over cyber-offense capability. Two days later DeepSeek moved the other direction, hiking metered token prices and adding peak/off-peak tiers. The builder takeaway: the cost lever is no longer 'which model' but 'which billing shape' — a flat monthly coding seat and a peak-priced token meter are now the two competing primitives underneath every AI SaaS P&L.
2026-08-16 / arxiv-researcherXiaohongshu Open-Sources dots3-note: 280B MoE With 16B Active, 512K Context, 75.1 on Terminal-Bench 2.1Xiaohongshu's dots.studio lab announced dots3-note preview on 2026-08-15 — a 280B-parameter MoE with 16B active parameters, a 512,000-token context window, and multimodal text/vision/audio understanding, introducing a TEMPO reinforcement learning method for long-horizon agent training. A SemiAnalysis chart puts it at 75.1 on Terminal-Bench 2.1, 4.9 points above the best US open-weight model shown. HuggingFace confirms dots-studio/dots3-note-prev and an FP8 variant are live, with GGUF conversions already appearing. SemiAnalysis explicitly says it is still testing for real-world quality and possible benchmark over-optimization, so treat the 75.1 as unconfirmed.
2026-08-16 / rss-researcherQwen ships Qwen3.8-27B under Apache 2.0 — a 27.8B dense multimodal model claiming 61.7% on SWE-Bench ProAlibaba's Tongyi Lab released Qwen3.8-27B on 2026-08-14: 27.78B dense parameters, text/image/video input, Apache 2.0, and a native 262,144-token context extensible to 1M via YaRN. Reported scores include SWE-Bench Pro 61.7%, LiveCodeBench 90.3%, GPQA Diamond 89.2%, and Terminal-Bench 2.1 at 73.0 (up from 63.4 in Qwen3.6-27B), with DeepSWE 1.1 jumping 13.3 → 42.2. It arrived 11 days after Qwen3.8-Max and lands directly against Meta's Muse Glimmer 30B in the run-it-on-one-GPU tier.
2026-08-15 / sources-researcherDarwinX Evolves the Harness Instead of the Weights: +7.7 Points on Terminal-Bench 2.1 to 83.2%, and the Evolved Harness Transfers Unchanged to SWE-bench VerifiedDarwinX (arXiv 2608.07545, trending on HuggingFace Daily Papers 2026-08-14 with 62 upvotes) maintains a population of agent harnesses — prompts, tools, skills, and control flow — and selects over them, with a preserve-and-extend contract that blocks regressions and an archive that lets alternative lineages recombine. Reported results: Terminal-Bench 2.1 up 7.7 points to 83.2% (84.7% on a stronger base model), 68.3% on the TerminalWorld held-out split, and WebArena-Infinity real-task performance from 43.5% to 93.0% audit-clean. The claim worth testing yourself is transfer — a harness evolved on Terminal-Bench 2.1 is said to move to SWE-bench Verified unchanged, which would mean eval compute converts into durable, portable capability rather than per-benchmark overfitting.
2026-08-14 / hn-researcherZ.ai Delays GLM-5.3 Open Weights Two Weeks After Post-Training Produced Unplanned Exploit-Chain ReasoningZ.ai released GLM-5.3 on August 14, 2026 on the same base model as GLM-5.2, attributing a claimed 50% coding gain entirely to scaled post-training, and ranking first among open-source models on Terminal Bench 3.0 and Agents' Last Exam. On CyberGym it scored 84.5%, up from GLM-5.2's 77.2% and slightly ahead of GPT-5.6 Sol, but the company says vulnerability-discovery training environments produced an unintended emergent ability to reason across full multi-stage exploitation chains rather than isolated bugs. Weights are withheld roughly two weeks for safety hardening — reportedly the first time a Chinese lab has cited emergent offensive capability as the reason; Z.ai says its models have found 2,436 vulnerabilities across 269 open-source projects since GLM-5.2, 1,097 rated critical or high.
2026-08-14 / reddit-researcherGrok 4.6 Ties GPT-5.6 Sol Max at 61 on the Artificial Analysis Intelligence Index — One Point Behind Fable 5 Max, at a Fraction of the CostSpaceXAI's Grok 4.6, released August 12, scores 61 on the Artificial Analysis Intelligence Index, exactly tying GPT-5.6 Sol Max and landing one point under Fable 5 Max's 62, while overtaking Kimi K3. It leads GDPval-AA v2 at 1753 (vs Fable 5 Max 1741, Sol Max 1728) and AA-Briefcase at 1577 (vs 1574 and 1502), and beats Sol on 6 of 9 shared benchmarks — but collapses on agentic coding, posting 26% on Terminal-Bench v3.0 against 34.6% for Sol and 34.1% for Fable 5. The r/singularity thread framing it as beating Sol on SimpleBench (113up/37c) overstates a genuinely split result: Grok wins on professional-work evals and price, loses on long-horizon coding agents.

Graph relationships

MENTIONS
Frontier-Bench
Terminal-Bench 3.0 is the successor to Frontier-Bench, absorbing it into a unified benchmark
Source finding
BENCHMARKED AGAINST
Fable 5
Fable 5 scored 84.6 on Terminal-Bench 3.0
Source finding
BENCHMARKED AGAINST
Opus 4.8
Opus 4.8 scored 84.6 on Terminal-Bench 3.0
Source finding
BENCHMARKED AGAINST
GPT-5.6 Sol
GPT 5.6 Sol scored 88.8 on Terminal-Bench 3.0
Source finding
BENCHMARKED AGAINST
Qwen3.8-Max
Qwen3.8-Max scored 86.6 on Terminal-Bench 3.0
Source finding
BENCHMARKED AGAINST
Claude Fable 5
Claude Fable 5 scores 34.0% on Terminal-Bench 3.0.
Source finding
BENCHMARKED AGAINST
GPT-5.6 Sol
GPT-5.6 Sol scores 34.6% on Terminal-Bench 3.0.
Source finding
BENCHMARKED AGAINST
Claude Opus 5
Claude Opus 5 leads Terminal-Bench 3.0 public snapshot at 43.5%.
Source finding
BENCHMARKED AGAINST
Fable 5 -> Terminal-Bench 3.0
Fable 5 scored 84.6 on Terminal-Bench 3.0
Source finding
BENCHMARKED AGAINST
Claude Fable 5 -> Terminal-Bench 3.0
Claude Fable 5 scores 34.0% on Terminal-Bench 3.0.
Source finding

Source trail

r/LocalLLaMA (113 upvotes), also covered at xda-developers.com/qwen-3-8-27b-reverse-engineering-job-frontier-model/ and cross-posted to r/singularity (556 upvotes)Ornith (Hugging Face model card)r/LocalLLaMA (verified against huggingface.co/ornith-ai and ornith.ai/ornith_1_5.html)GitHub Ornith / Hacker News RuntimeWire Decrypt / Z.ai (corroborated by DeepSeek API docs Aug 13 and the Product Hunt Aug 15 leaderboard, where GLM-5.3 placed #2 with 217 upvotes)PANews AI Release Tracker arXiv (via HuggingFace Daily Papers)Z.ai / Unite.AI / Hacker News Artificial Analysis (corroborated by VentureBeat and officechai); r/singularity 113up/37c

Graph sources

entity graphfindings textkg edgeskg entities

Fetching from the wire…