Sebastian Raschka: Open Weights Are Becoming Infrastructure, and He Reverse-Engineered Kimi K3's Architecture Within an Hour of the Weights Dropping
In an August 3 conversation with Hugo Bowne-Anderson, Raschka argued open-weight models are shifting from research curiosity to infrastructure: 'It would be sad if local models were not an option and there were only proprietary models. It's good to have alternatives.' Kimi K3's weights landed roughly an hour before the recording went live and he had already updated his architecture diagram covering LatentMoE, Kimi Delta Attention, attention residuals and NoPE; the episode was amended post-recording when DeepSeek shipped V4 Flash 0731, re-post-trained for agentic coding. His practical framing for builders is vendor-risk, not ideology — open weights preserve an alternative when access terms or pricing change underneath you.
↳ Follow the thread