Public story · 2026-02-22 · source-backed
Arcee Trinity Large: Open 400B Sparse MoE (arXiv 2602.17004)
Story
Largest open-source sparse MoE model: 400B parameters, 13B active per token, trained on 17 trillion tokens. Novel SMEBU for load balancing. Zero loss spikes with Muon optimizer across 17T tokens — a remarkable training stability achievement. arXiv | Arcee Blog
Related stories
Each link below shares sources, entities, or timing with this story.
Shared entities / Same source / Shared topic / What happened next
Research Papers Worth Reading
Both cover Arcee Trinity Large, Largest, Sparse MoE; cite the same source (arXiv, Arcee Blog); overlapping topics (model, sparse).
Shared entities / Same source
Best Content This Week
Both cover Arcee Blog, Arcee Trinity Large; cite the same source (arXiv, Arcee Blog).
Shared entity: MoE / Shared topic / What happened next / Tension
An Open-Source Model Just Beat GPT-5.4 and Opus 4.6 on the Hardest Coding Benchmark
Both cover MoE; overlapping topics (active, model, parameter); picks up the MoE thread on 2026-04-20.
Shared entity: MoE / Shared topic / What happened next
DeepSeek V4 Drops: MIT-Licensed Frontier Model at 1/6th the Cost, Running on Chinese Chips
Both cover MoE; overlapping topics (active, model, parameter, token); picks up the MoE thread on 2026-04-26.
Shared entity: MoE / Same source domain / Shared topic / What happened next
Nemotron-Cascade 2: 30B MoE Delivers Best-in-Class at 3B Active Params
Both cover MoE; reported by the same outlet (arxiv.org); overlapping topics (active, model).
Shared entity: Muon / Shared topic / What happened next
PRX Part 3: Text-to-Image in 24 Hours
Both cover Muon; overlapping topics (model, muon, optimizer, token); picks up the Muon thread on 2026-03-04.
Shared entity: Muon / Same source domain / What happened next / Downstream implication
Muown implicitly performs angular step-size decay during pretraining.
Both cover Muon; reported by the same outlet (arxiv.org); picks up the Muon thread on 2026-06-23.
Shared entity: MoE / Shared topic / What happened next / Tension
Zyphra's ZAYA1-8B was trained from scratch on AMD Instinct, not NVIDIA.
Both cover MoE; overlapping topics (active, model); picks up the MoE thread on 2026-06-12.
Source trail
Entities
Provenance
- Canonical issue
- Ramsay Research Agent — 2026-02-22
- AI generated
- no
- Story unit
- 2026-02-22-arcee-trinity-large-open-400b-sparse-moe-arxiv-2602-17004
- Labels
- source-backed, canonical briefing excerpt