Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-17 · high
Three model sizes, trained from scratch on an 88B-token K-5 corpus, ran against matched unfiltered controls.
Why now: The paper's Hacker News post had reached 241 points by August 17.
A research team trained AI models from scratch on an 88-billion-token diet of elementary-school text, per the LittleLearner project. No amount of fine-tuning pushed the models past what that diet left out. For anyone counting on post-training to add a missing skill, that's a hard boundary, not a workaround waiting to be found.
The team, seven researchers from the Max Planck Institute for Intelligent Systems, ELLIS Tübingen and ETH Zürich, built the training set themselves. They call it LittleCurriculum: 88 billion tokens pulled from FineWeb-Edu and run through a five-stage filter matched to the Common Core K-5 curriculum.
From that they trained 0.6B, 1.3B and 5B parameter models from scratch, alongside matched control models trained on unfiltered data of the same size.
Then they tried to push the K-5-only models past their training. They scaled the models up, ran supervised fine-tuning plus GRPO reinforcement learning, and tested in-context learning with examples in the prompt.
The project's own summary put it plainly: "scaling, SFT+GRPO post-training, and in-context learning amplify what the curriculum taught, but none meaningfully improves out-of-scope performance." The models got better at elementary-school-level tasks. They didn't get better at anything outside that scope.
If this holds at larger scale, prompt engineering and reinforcement learning are bounded by what's in the pretraining mix, not by skill applied afterward.
Each link below shares sources, entities, or timing with this story.
Unsloth released GRPO / Shared entity: GRPO / Earlier coverage / Tension
Linked by a graph relationship (Unsloth released GRPO); both cover GRPO; earlier GRPO coverage from 2026-06-20.
Unsloth released GRPO / Shared entity: RLHF / Earlier coverage
Linked by a graph relationship (Unsloth released GRPO); both cover RLHF; earlier RLHF coverage from 2026-06-23.
DeepSeek-R1 uses GRPO
Linked by a graph relationship (DeepSeek-R1 uses GRPO).
DPO competes with RLHF / Shared entities / Earlier coverage
Linked by a graph relationship (DPO competes with RLHF); both cover GRPO, SFT; earlier GRPO coverage from 2026-07-18.
GRPO competes with PPO / Shared entities / Earlier coverage
Linked by a graph relationship (GRPO competes with PPO); both cover GRPO, SFT; earlier GRPO coverage from 2026-07-31.
Unsloth released GRPO / Shared topic
Linked by a graph relationship (Unsloth released GRPO); overlapping topics (clean, model).
Linked by a graph relationship (Unsloth released GRPO); overlapping topics (builder, model).
DPO competes with RLHF / Shared entity: RLHF / Earlier coverage / Downstream implication
Linked by a graph relationship (DPO competes with RLHF); both cover RLHF; earlier RLHF coverage from 2026-05-11.