The Stack v3: 4.9 Trillion Tokens of Code Across 713 Languages, and This Time the File Contents Are Actually in the Dataset
Elevated by The Batch #365, The Stack v3 ships as two artifacts: a training subset of 15.9 TB / ~4.9T tokens across 713 languages from 173M repositories, and a full pre-filter corpus of 113.7 TB across 770 languages from 224M repositories, with a crawl cutoff of August 7, 2025 and an ODC-By v1.0 license (source files keep their original licenses). The scale jump over v2 is roughly 9x in training tokens (550B → 4.9T), but the operationally important change is that file contents are embedded inline rather than referenced by file ID, making the dataset self-contained instead of requiring a separate content fetch. Dedup moved to language-agnostic near-dedup with Jaccard verification after connected-component clustering, rows are grouped one-per-repository for full-repo-context pretraining, and opt-outs via the 'Am I in The Stack?' Space are removed before each patch release; the train split has 182,428 downloads in the last month.
↳ Follow the thread