Import AI #457: Muon Optimizer Suffers 25%+ Neuron Death During Warmup — Aurora Alternative Achieves Lower Loss on 1.1B Params

Import AI·medium signal

Tilde Research identified a critical flaw in the Muon optimizer: over 25% of neurons become inactive during warmup, a problem they term 'neuron death.' Their alternative, the Aurora optimizer, tested on 1.1B-parameter transformers achieves loss of 2.26 vs Muon's 2.31 and a 10-point MMLU improvement. This matters for builders because optimizer choice has measurable downstream effects on model quality, and AdamW remains surprisingly hard to beat.