xHC Scales Transformer Residual Streams to 16 Parallel Paths — 4.0-Point Downstream Gain on an 18B MoE

arXiv / HuggingFace Daily Papers (rednote-hilab)·medium signal

Hyper-Connections expand a Transformer's residual stream into N parallel streams but had stalled at N=4 due to weak information flow and cubic cost. xHC (arXiv 2607.14530, July 16, from Xiangdong Zhang and 12 co-authors) reaches N=16 by sparsely updating only k=4 streams plus temporal feature augmentation, scoring 4.0 points higher on average downstream than the prior mHC on an 18B MoE. Vanilla and mHC need 1.50× and 1.19× xHC's compute to hit the same loss, and an xHC-Flash variant cuts memory traffic from 73.5C to 40C per sublayer.