Mamba-3: Inference-First State Space Model Beats Transformers 4%, Runs 7x Faster at Long Sequences

Together.ai·medium signal

Together.ai released Mamba-3 (Apache 2.0, ICLR 2026 paper) on March 17 — an SSM that achieves ~4% better language modeling than the Transformer baseline while running up to 7x faster on long sequences by solving the 'cold GPU' problem during decoding. Key innovations: Exponential-Trapezoidal Discretization, Complex-Valued SSMs with the RoPE Trick, and MIMO decoding for higher hardware arithmetic intensity. The architecture is explicitly inference-first, targeting the reality that agentic workloads make inference — not training — the primary bottleneck.