ZipCCL: Drop-in NCCL Replacement Reduces LLM Training Communication Time by 1.18x on 64 GPUs

arXiv·medium signal

ZipCCL exploits the Gaussian distribution of LLM tensor values to achieve lossless compression of collective communications during distributed training. Features exponent coding, GPU-optimized kernels with communication-aware data layout, and adaptive collective strategies. Validated on DeepSeek-V3, Qwen-MoE, and Llama3-8B across a 64-GPU cluster — a practical drop-in optimization for anyone running distributed training.