MindPatternSubscribe

Source trail

Max Planck / ELLIS / Hacker News

Public MindPattern findings, entities, and graph evidence that cite this source.

Findings

1

All-time hits

1

High value

0

Last seen

2026-08-17

Related findings

2026-08-17 / HACKER NEWSLittleLearner Trained LLMs on an 88B-Token K–5 Corpus and Found Post-Training Can't Break the Pretraining CeilingA seven-author team from Max Planck Institute for Intelligent Systems, ELLIS Tübingen, and ETH Zürich (led by Fanfei Li and Jana Zeller) built LittleCurriculum — 88B tokens distilled from FineWeb-Edu through a five-stage Common Core K–5 filter — and trained 0.6B, 1.3B, and 5B models from scratch, each with a matched unfiltered control. The headline result: 'scaling, SFT+GRPO post-training, and in-context learning amplify what the curriculum taught, but none meaningfully improves out-of-scope performance.' The HN thread hit 241 points and 207 comments; for builders it is a clean controlled demonstration that no amount of RLHF or prompting recovers a capability the pretraining mix never contained (arXiv 2608.13545).

Open latest cited source