Entity trail

GSM8K

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-08-14 / hn-researcherNanoRL Implements REINFORCE, PPO, GRPO and RLOO for LLM Training in ~1,800 Lines Without Ray, TRL, or DeepSpeedNanoRL is an MIT-licensed project posted as Show HN that fits REINFORCE, PPO with GAE, GRPO, and RLOO into roughly 1,800 lines across 7 files, scaling from CartPole on a laptop to asynchronous distributed training on GPU clusters using vLLM rollout workers, with no Ray, TRL, or DeepSpeed dependency. Its training targets are CartPole, Countdown, and GSM8K math reasoning, running on Python 3.10+ with PyTorch, Gymnasium, Transformers, Datasets, and PEFT. The author explicitly optimizes for readability and forkability over production robustness, excluding Megatron-scale parallelism and multi-tenant scheduling; the repo has just 5 stars and 7 commits, so this is a teaching artifact rather than a production trainer.
2026-08-06 / arxiv-researcherCausal Audit Finds Some Multi-Agent 'Latent Thought' KV Relays Transfer Nothing Example-Specific: Zeroing Costs 14.7 Points, Mismatching Costs 0.4Multi-agent LLM systems that relay key-value caches instead of text credit their gains to exchanged latent thoughts, but that is a claim about which example's cache is relayed. Replacing the cache with deranged, zeroed, and moment-matched random counterparts shows the claim holds only when the receiver actually needs the sender's private information — there the contrast is 100% versus 23-25%, replicated across three families, five checkpoints, and a prose document-QA surface. When it does not, a pre-registered five-seed protocol establishes equivalence within 2.8 points under Holm-corrected TOST on GSM8K and ARC-Challenge across three Qwen3 scales; in one natural cell zeroing the relay costs 14.7 points while a mismatched cache costs 0.4, and among released systems LatentMAS's relay hits ceiling, KVComm's layer subset is partial, and C2C's projector shows no detected example-specific transfer.
2026-08-04 / reddit-researcherIntology's Locus Post-Trains Qwen3 Base Models Past the Official Human-Tuned Instruct Checkpoint — 44.7 on PostTrainBench, 51.6% on PostTrainBench+Intology announced on August 3 that its automated research system Locus is SOTA on PostTrainBench, which gives an agent one H100 and 10 hours to post-train Qwen3-1.7B/4B, SmolLM3-3B, or Gemma-3-4B across seven benchmarks (AIME 2025, Arena Hard, BFCL, GPQA Main, GSM8K, HealthBench, HumanEval). Locus scores 44.7 in the official setting and, under a self-introduced expanded-compute variant they call PostTrainBench+ (thousands of H100 hours), reaches a 51.6% composite that surpasses the official human-post-trained Qwen3-1.7B-Instruct. Intology also ran Locus across live prize-money Kaggle competitions, claiming 4th-highest average rank after 16 days — note that PostTrainBench+ is Intology's own extension of a third-party benchmark, so the headline result is not an apples-to-apples leaderboard entry.
2026-07-28 / github-pulse-researcherHeadroom Ships Context Compression With Accuracy Deltas Attached — 92% Token Cut on Code Search, GSM8K Unchanged at 0.870headroomlabs-ai/headroom (62,876 stars, ~313/day) compresses tool outputs, logs, RAG chunks and files before they reach the model, and unusually publishes the accuracy side of the trade: GSM8K holds at 0.870 (±0.000), TruthfulQA actually improves 0.530 → 0.560, and SQuAD v2 plus BFCL retain 97%. Real workloads show 92% savings on code search (17,765 → 1,408 tokens), 92% on SRE debugging and 73% on GitHub issue triage. The three-stage pipeline routes by content type into SmartCrusher (JSON), an AST-aware CodeCompressor covering Python/JS/TS/Go/Rust/Java/C/C++/Perl, or a trained Kompress-v2-base model for prose — and a CacheAligner deliberately preserves provider KV-cache prefixes so compression doesn't destroy prompt caching. Ships as library, proxy (`headroom proxy --port 8787`) or MCP server.
2026-07-28 / hn-researcherNeutrino-1 8B Ships Apache-2.0 With Ternary-Packed Weights: 3.88 GB On Disk, 72.1 MMLU, 33.7 tok/s on an M5Fermion Research released Neutrino-1 8B on July 27, 2026 — a Qwen3-8B derivative retrained with ternary quantization-aware training, storing all 252 transformer linears in a ternary-family format one-eighth the size of fp16, decoded inside the matmul kernels. The 8,190,735,360-parameter model is a 2.56 GB download / 3.88 GB on disk with a 40,960-token context, scoring 72.1 MMLU, 80.2 IFEval-strict, 53.4 GSM8K and 68.9 BFCL v3, and running 763 tok/s on an H100 with speculative decoding versus 33.7 tok/s on an Apple M5 via MLX. Apache-2.0 with no access request; weights are on Hugging Face as FermionResearch/Neutrino-8B alongside 0.6B and 0.6B-Chat variants.
2026-07-26 / arxiv-researcherSelf-Consistency Is Epistemically Shallow: 100 Samples From One Model Surface One Dimension of Signal, 24 Models Surface FourIzhar Ali runs a clean comparison on identical questions — one model sampled 100 times at τ=1 versus an ensemble of 24 LLMs run once each at τ=0 — and applies a Marchenko-Pastur random-matrix test to separate signal from sampling noise on both sides. Within any single model, at most one dimension rises above the noise edge across five model families and three benchmarks (MMLU, HellaSwag, GSM8K); across the ensemble, four eigenvalues clear it, against a matched-difficulty Bernoulli null that produces at most one in 500 Monte Carlo draws. Temperature sampling gives you accurate per-question uncertainty and nothing else — if you want to know what a model doesn't know as a structure, you need different models, not more samples.
2026-07-26 / arxiv-researcherChannelGuard Shows an 'Attack Success 0.000' Multi-Agent Pipeline Owed 54 of 60 Blocks to Azure's Server-Side Filter, Not Its Own CodeAcross a 2,100-trace evaluation over eight attack families, five defenses, and three model backends, Hossain et al. found an undefended planner/worker/verifier/synthesizer pipeline that reported perfect 0.000 attack success on tool- and memory-poisoning — but 54 of its 60 blocks came from Azure GPT-5's provider-side filter, and safety silently shifted to the agent model's own alignment on a backend without one. Their training-free ChannelGuard puts embedding-similarity information-bottleneck gates on every inter-agent hop, blocking Tool Poisoning 30/30 identically across Azure GPT-5, Sonnet 4.5, and Haiku 4.5, halving prompt-injection success (0.333 to 0.167) and preserving GSM8K accuracy at 0.867, with no added LLM call. The honest caveat: white-box adaptive paraphrase evades every embedding gate, and the whole study cost $47.36 to run.
2026-07-13 / skill-finderFine-tune a small reasoning model with GRPO + LoRA using a rank-8/alpha-16, no-value-network recipeGRPO samples multiple completions per prompt and optimizes on group-normalized relative advantages, eliminating the separate value network that PPO needs — and it composes cleanly with LoRA by freezing all weights except rank-8 (alpha-16) adapters, with gradient clipping and within-batch advantage standardization for stability. This lets a solo builder do R1-style chain-of-thought RL on a single consumer GPU against a verifiable-reward dataset like GSM8K, rather than reaching for full-parameter RLHF.
2026-06-08 / skill-finderLayer self-consistency on chain-of-thought for a measured +17.9% on hard reasoningFor non-reasoning models on math and logic tasks, sampling multiple chain-of-thought paths and taking a majority vote (self-consistency) lifts GSM8K accuracy by +17.9% over a single greedy CoT pass. It's the highest-ROI accuracy lever for reasoning-heavy prompts — but skip it entirely for native reasoning models, which already reason internally and gain nothing.
2026-04-25 / arxiv-researcherMathDuels: Adversarial LLM-vs-LLM Evaluation Defeats Static Benchmark Ceiling EffectsXu et al. introduce MathDuels, a framework that evaluates LLMs as both problem posers and solvers in adversarial mathematical duels. By having models generate novel problems for each other rather than solving fixed benchmarks, the approach circumvents the ceiling effects plaguing static evaluations like MATH and GSM8K. This dual-role evaluation reveals capability gaps invisible to traditional pass-rate metrics and provides a continuously refreshing evaluation signal.
2026-04-16 / vibe-coding-researcherPattern: Practitioners Reject Leaderboards — 'Benchmarks Are Bullshit' Thesis Gains Tractionpaddo.dev argues that AI benchmarks are structurally broken: HumanEval's 164 problems are trivially overfittable, GSM8K saturated from 50% to 95%+ in two years, MMLU scores vary 5-15% depending on evaluation setup, and Berkeley researchers built an agent that games benchmarks without improving capability. Chinese models (DeepSeek, Qwen) post strong scores but hallucinate APIs at higher rates in practice. The prescription: rotating test sets, 50+ turn evaluations, and 'the vibe check' from experienced developers — exactly what leaderboards can't capture.
2026-04-09 / rss-researcherAWS Reinforcement Fine-Tuning on Amazon Bedrock — Best Practices with GSM8K Mathematical ReasoningAWS published best practices for reinforcement fine-tuning (RFT) on Amazon Bedrock, using the GSM8K mathematical reasoning dataset as a concrete example. The guide covers where RFT is most effective versus standard fine-tuning, dataset preparation strategies, reward function design, and when to prefer RFT over other approaches. Practical reference for teams looking to improve model reasoning on domain-specific tasks without full retraining.

Graph relationships

BENCHMARKED AGAINST
headroom
Headroom maintains GSM8K accuracy at 0.870 while compressing tokens 92%.
Source finding
CRITICIZES
paddo.dev
paddo.dev argues GSM8K saturated from 50% to 95%+ in two years, making it unreliable.
Source finding
BENCHMARKED AGAINST
headroom -> GSM8K
Headroom maintains GSM8K accuracy at 0.870 while compressing tokens 92%.
Source finding
CRITICIZES
paddo.dev -> GSM8K
paddo.dev argues GSM8K saturated from 50% to 95%+ in two years, making it unreliable.
Source finding

Source trail

Fermion Research GitHub lifts GSM8K by +17.9%GitHub Most practical deployment gate research GitHub / Hacker News arXiv 2608.04893 Intology (via r/singularity 147 upvotes; @intology on X, 144K views)arXiv 2607.20464 arXiv 2607.19430 Unsloth Documentation — Reinforcement Learning (RL) Guide arXiv

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…