Five Startups Are Betting Against the Transformer — and Two Now Claim Parity With Mainstream LLMs
MIT Technology Review profiled (Aug 10) five labs replacing or bypassing attention: Subquadratic's SubQ (sparse attention it claims is the first to rival top mainstream LLMs on search and coding), Manifest AI's power retention (a drop-in swap where flash_attention becomes power_retention, claiming >10x training and >100x inference speedups at 64k context, shipped as PowerCoder and Brumby), Liquid AI's 20%-transformer/80%-liquid hybrid (34 million downloads, matching rivals 4x bigger, running on a $50 Raspberry Pi), Inception's diffusion-based Mercury 2 (GPT-4-class output ~10x faster), and Pathway's state-space Dragon Hatchling (solved >97% of 250,000+ sudoku puzzles where competing models failed all of them). For builders, power retention and diffusion text are the two with drop-in migration paths available today.
↳ Follow the thread