Fetching from the wire…
01
02
03
04
05
06
07
08
Infra2026-08-27 · source-backed
Three new components ship with it: GatedResidual, which mixes multiple residual streams with elementwise gating before each attention and MoE block; Qwen Sparse Attention, which scores compressed key blocks and selects contiguous token blocks while leaving the trailing block uncompressed; and Per-Layer Embedding, enriching selected decoder layers with hashed token n-gram features (GitHub). Combined with Gated DeltaNet, QSA makes Qwen4-Exp the first architecture integrating both linear and sparse attention. The block-level selection targets long-context inference cost, which is the binding constraint on agent workloads.
Each link below shares sources, entities, or timing with this story.
Shared entities / Same source domain / Shared topic
Both cover Gated DeltaNet, GitHub, MoE, Qwen4; reported by the same outlet (github.com); overlapping topics (architecture, attention).
Shared entities / Shared topic / Earlier coverage
Both cover Combined, Gated DeltaNet; overlapping topics (architecture, attention, combined, linear, token); earlier Combined coverage from 2026-03-07.
Shared entities / Same source domain / Shared topic / Earlier coverage
Both cover Combined, GitHub; reported by the same outlet (github.com); overlapping topics (agent, combined).
Both cover GitHub, MoE; reported by the same outlet (github.com); overlapping topics (agent, token).
Both cover GitHub, MoE; reported by the same outlet (github.com); overlapping topics (agent, architecture).
Shared entity: GitHub / Same source domain / Shared topic / Earlier coverage / Tension
Both cover GitHub; reported by the same outlet (github.com); overlapping topics (agent, architecture, token).
Both cover GitHub; reported by the same outlet (github.com); overlapping topics (agent, architecture, block).
Shared entities / Shared topic / Earlier coverage / Tension
Both cover Combined, GitHub; overlapping topics (architecture, combined); earlier Combined coverage from 2026-08-25.