Kimi K3 is Kimi Linear scaled from 48B to 2.8T, and the first frontier model to drop RoPE entirely.

Sebastian Raschka's July 28 teardown argues K3 is less exotic than the release framing suggests: a scaled production version of Kimi Linear with Kimi Delta Attention as the hybrid attention layer and LatentMoE compressing large linear layers by down-projection. The genuinely new piece is replacing every RoPE layer with NoPE, the first frontier-scale no-positional-embeddings implementation, alongside attention residuals that weight cross-layer residual connections by attention scores for roughly 4% added training cost and 2% added inference cost. He groups K3 with Nemotron 3 Ultra and DeepSeek V4 as a clear industry pivot toward inference efficiency.