Apple ParaRNN Achieves 665x Speedup Over Sequential RNN Training — First 7B Classical RNN Competitive with Transformers (ICLR 2026 Oral)

Apple Machine Learning Research·medium signal

Apple's ParaRNN framework casts nonlinear recurrence as a single system of equations solved via Newton's iterations in parallel, achieving a 665x speedup over sequential RNN training. This enabled training the first 7-billion-parameter classical RNNs competitive with transformers on language modeling. Accepted as an ICLR 2026 Oral presentation with code publicly released.