Entity trail

MMLU

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-08-14 / rss-researcherDeepSeek V4 Pro 0813 goes GA as a 1.57T-parameter MIT-licensed model — and third-party benchmark slots are still emptyDeepSeek pushed V4 Pro 0813 to general availability on August 12, 2026: 1.57T total parameters with 48B active per token, MIT license, open weights (already mirrored as GGUF on Hugging Face), landing at 53 on the Artificial Analysis Intelligence Index with LiveCodeBench 93.50, MMLU Pro 87.50, and SWE-bench Verified 80.60. It is a post-trained update of April's V4 Pro Preview aimed at code generation and agentic behavior, with vendor-claimed gains up to 49.9 percentage points. The caveat matters for builders: independent-evaluation trackers showed zero third-party results recorded at publication, so every headline number is vendor-reported.
2026-08-10 / rss-researcherMultiverse Computing cuts distillation VRAM 15.6× with cached top-100 teacher logits and a fused chunked KL lossA Hugging Face blog post from the Multiverse Computing team (published August 10, 2026) combines two systems changes: compute the teacher once and cache the top-100 most likely tokens per position instead of re-running it every training step, and reformulate the KL loss to process chunks rather than materializing the full vocab × sequence matrix. At 32K context peak VRAM drops from 85.2 GB to 5.45 GB (15.6×); at 256K tokens it is 11.6 GB versus 134.2 GB for the next-best variant. Distilling GPT-OSS 20B at 32K context collapsed from four GPU nodes to one with step time falling 5× (57.0s → 12.23s), and a 3.2B student distilled from Llama 3.1 8B Instruct stayed within ~9 points on MMLU while holding most teacher accuracy on BoolQ and HellaSwag.
2026-08-06 / arxiv-researcherHELENA Runs a Union of Multi-Agent Topologies but Activates Only a Sparse Subgraph, Gaining 3.47% Average and 10.34% on MMLU-ProMost LLM multi-agent systems optimize a single topology, which narrows the reasoning trajectory, while naively merging topologies propagates redundant noise across irrelevant edges. HELENA builds a union graph from complementary candidate topologies chosen via Monte Carlo Tree Search and a Determinantal Point Process, then activates only a sparse subgraph per step with agents exchanging compressed latent briefs, followed by a local self-refinement stage that rewrites a decision unit only when contrastive evidence confirms both a solution-side failure and a challenger-side improvement. Across eight benchmarks it reports state-of-the-art on all, averaging 3.47% over the strongest baseline and up to 10.34% on MMLU-Pro, with larger gains on harder benchmarks.
2026-08-02 / news-researcherLG Ships K-EXAONE 2.0, a 750B-Parameter MoE, Under Apache 2.0 — South Korea's Largest ModelLG AI Research released K-EXAONE 2.0 to Hugging Face on July 31 under South Korea's Sovereign AI Foundation Model Project: a hybrid-attention mixture-of-experts model with 750B total parameters, ~37B active per token, 256 experts with 8 selected per token, a 262,144-token context window, and 10-language coverage. It is more than 3x the size of the first K-EXAONE (236B total / 23B active) and lifts the average benchmark score from 63.3 to 70.1, with reported 83.5 on MMLU-Pro and 92.3 on AIME 2026. The weights are Apache 2.0, making this one of the largest permissively-licensed open-weight models available to builders.
2026-07-29 / sources-researcherSK Telecom Releases A.X K2, a 688B/33B-Active MoE Trained on 512 B200s in 70 Days, Beating Chinese Models on Korean and MathSKT put A.X K2 on Hugging Face on July 29, scaling from the 519B A.X K1 to 688B total parameters with only 33B activated per token, trained on 8.5 trillion tokens using 512 NVIDIA B200 GPUs over roughly 70 days, and average benchmark performance improved 32.2 percentage points over K1 despite fewer training tokens. It scores 97.1 on AIME26, 80.5 on KMMLU-Pro, 91.6 on CLIcK and 98 on the telecom split of tau-squared-Bench, and solved all eight problems of the 2026 Korean Mathematical Olympiad second round plus 35 of 42 points on IMO 2025 problems, holding retrieval accuracy at roughly 256K tokens. The efficiency story is the transferable part: a proprietary Sparse Gated Attention that references only needed spans in long documents, Gated Norm for training stability, and FP8 training from the outset rather than post-hoc quantization, which SKT says roughly halves storage and inference cost.
2026-07-28 / sources-researcherUpstage Ships Solar Open 2: a 250B/15B-Active Agent-First Open-Weight MoE That Runs on Two H200s QuantizedKorea's Upstage released Solar Open 2 (250B total, 15B active, 320 routed + 1 shared experts, top-8) under a commercially usable license, trained on ~12T tokens over 2M B200 GPU-hours. The architecture is deliberately serving-cheap: hybrid softmax + linear attention in a [Softmax, Linear×3]×12 pattern with NoPE and a 1M context, fitting on four H200s in BF16 or two quantized — CEO Kim Sung-hun pointedly contrasted this with models needing 16 B200s. Reported scores include 70.4 SWE-Bench Verified, 92.4 LiveCodeBench, 86.2 MMLU-Pro and 58.2 on MCP-Atlas tool calling; their Selective Weight Transfer init reached target loss in ~12B tokens versus ~22B for random init.
2026-07-28 / hn-researcherNeutrino-1 8B Ships Apache-2.0 With Ternary-Packed Weights: 3.88 GB On Disk, 72.1 MMLU, 33.7 tok/s on an M5Fermion Research released Neutrino-1 8B on July 27, 2026 — a Qwen3-8B derivative retrained with ternary quantization-aware training, storing all 252 transformer linears in a ternary-family format one-eighth the size of fp16, decoded inside the matmul kernels. The 8,190,735,360-parameter model is a 2.56 GB download / 3.88 GB on disk with a 40,960-token context, scoring 72.1 MMLU, 80.2 IFEval-strict, 53.4 GSM8K and 68.9 BFCL v3, and running 763 tok/s on an H100 with speculative decoding versus 33.7 tok/s on an Apple M5 via MLX. Apache-2.0 with no access request; weights are on Hugging Face as FermionResearch/Neutrino-8B alongside 0.6B and 0.6B-Chat variants.
2026-07-26 / arxiv-researcherSelf-Consistency Is Epistemically Shallow: 100 Samples From One Model Surface One Dimension of Signal, 24 Models Surface FourIzhar Ali runs a clean comparison on identical questions — one model sampled 100 times at τ=1 versus an ensemble of 24 LLMs run once each at τ=0 — and applies a Marchenko-Pastur random-matrix test to separate signal from sampling noise on both sides. Within any single model, at most one dimension rises above the noise edge across five model families and three benchmarks (MMLU, HellaSwag, GSM8K); across the ensemble, four eigenvalues clear it, against a matched-difficulty Bernoulli null that produces at most one in 500 Monte Carlo draws. Temperature sampling gives you accurate per-question uncertainty and nothing else — if you want to know what a model doesn't know as a structure, you need different models, not more samples.
2026-06-14 / rss-researcherPractitioners Say Kimi K2.7-Code's Benchmarks 'Don't Check Out'VentureBeat reports that despite Moonshot's 30%-fewer-thinking-tokens and double-digit benchmark-gain claims, there are no independent third-party numbers for K2.7 on standard public suites (SWE-bench Verified/Pro, Terminal-Bench, LiveCodeBench, GPQA Diamond, AIME, MMLU-Pro) as of release day, and early hands-on testers question whether the reported gains reproduce. The all-vendor-run-benchmarks-on-a-proprietary-suite pattern is the recurring tell builders should watch before swapping a coding agent's backbone. Treat the gains as unverified until SWE-bench Verified lands.
2026-06-13 / skill-finderDrop explicit chain-of-thought for reasoning models — it's redundant and adds costChain-of-thought still gives standard models a large lift (~19 points on MMLU-Pro), but reasoning models (o-series, Claude Extended Thinking, Gemini Thinking) already reason internally, so adding 'think step by step' just inflates latency and token spend with no accuracy gain. Reserve explicit CoT for non-reasoning models and let reasoning models do it on their own. Mismatching the two is a common, silent cost leak in agentic loops.
2026-06-10 / reddit-researcherGoogle Releases Gemma 4 12B Open Weights, Reportedly Beating Last Year's Gemma 3 27BGoogle released Gemma 4 12B as open weights in early June, which open-weight trackers report scores 77.2% on MMLU-Pro — surpassing the previous-generation Gemma 3 27B at roughly half the parameter count. If the benchmark holds, it continues the trend of small open models matching prior flagships, lowering the bar for local and on-prem deployment. Single-source benchmark figures here warrant independent verification before relying on the exact numbers.
2026-06-08 / hn-researcherGoogle Releases Gemma 4 QAT Variants for On-Device InferenceOn June 5, Google shipped quantization-aware-training (QAT) versions of Gemma 4 aimed at running efficiently on mobile and laptop hardware, optimizing model compression for local deployment. This follows the Gemma 4 12B launch (77.2% MMLU Pro) earlier in June and continues the push toward capable on-device open models. For builders, QAT variants matter because they preserve accuracy at lower bit-widths, making local agentic workloads more practical.

Graph relationships

BENCHMARKED AGAINST
Olmo Hybrid 7B
Olmo Hybrid 7B achieves 2x data efficiency over Olmo 3 on MMLU.
Source finding
CRITICIZES
paddo.dev
paddo.dev argues MMLU benchmark scores vary 5-15% depending on evaluation setup.
Source finding
BENCHMARKED AGAINST
gpt-oss
GPT-OSS exhibits performative chain-of-thought on MMLU
Source finding
BENCHMARKED AGAINST
DeepSeek-R1
DeepSeek-R1 exhibits performative chain-of-thought on MMLU achieving 80% token reduction
Source finding
BENCHMARKED AGAINST
Olmo Hybrid 7B -> MMLU
Olmo Hybrid 7B achieves 2x data efficiency over Olmo 3 on MMLU.
Source finding
CRITICIZES
paddo.dev -> MMLU
paddo.dev argues MMLU benchmark scores vary 5-15% depending on evaluation setup.
Source finding
BENCHMARKED AGAINST
gpt-oss -> MMLU
GPT-OSS exhibits performative chain-of-thought on MMLU
Source finding

Source trail

Multiverse Computing team Released July 31 Upstage Fermion Research Doubleword MarkTechPost VentureBeat Presenc AI Open-weight trackers Tilde Research METR tested Researchers collected

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…