Tencent open-sources Hy4-preview, a 770B-parameter MoE with 49B active and a 1M-token context window

GitHub·high signalXBlueskyLinkedInCopy link

Tencent Hunyuan released and open-sourced Hy4 preview on 2026-08-28: 770B total parameters with 49B activated per token, 78 layers (77 of them MoE with 256 routed plus 1 shared expert, top-8 routing), a 1M context length, and a native 10B MTP layer for speculative decoding. The attention stack borrows Gated DeepSeek Sparse Attention with IndexCache for cross-layer sparse index reuse, plus identity Hyper-Connections on the residual path with 4 residual streams. Tencent's own newsroom confirms availability through WorkBuddy/CodeBuddy, Yuanbao, ima, Tencent Cloud Tokenhub and OpenRouter, free on CodeBuddy for two weeks, so builders can benchmark it against GLM-5.3 and Qwen4 without hosting 770B weights.