hfviewer's 'Expert Atlas' Traces All 896 Kimi K3 Routed Experts and Puts the Active Fraction at 1.8%
The r/LocalLLaMA post announcing K3 on hfviewer.com (160 upvotes) points to an analysis that maps every one of the 896 routed experts from the shipped weights, naming recurring specializations including a 'decorator expert,' a 'country radical' expert, and a catch-all sink. The accompanying plot traces expert-pool size against active fraction across recent MoEs and lands K3 at 1.8% active — the largest expert pool ever shipped in the open, and a continuation of the trend where pools grow while activation share falls. The closest previously traced relative is Kimi K2.7-Code, an MLA MoE routing top-8 of 384 experts plus one shared, which makes the K3 diff legible: three quarters of attention layers become KDA blocks and the expert pool more than doubles.
↳ Follow the thread