Abstract
Large multi-node GPU jobs often become limited by the number of bytes injected into the inter-node network path. Compression can reduce this traffic, but existing compression-assisted collective libraries either sit outside NCCL’s GPU-centric execution model or mainly rely on quantization alone. We present NCCLZ, a compression-enabled extension to NCCL that decouples lossy quantization from lossless entropy coding. NCCLZ performs approximation once before communication by converting tensors into integer symbols, and then applies lossless byte reduction inside NCCL’s communication pipeline. This placement keeps numerical approximation explicit, avoids repeated lossy recompression during reductions, and preserves NCCL’s native pipelined execution model. NCCLZ combines a runtime selector with slot-aware framing and pipelined staging to choose among raw transmission, fixed-length packing, and GPU Huffman coding. Initial results on Polaris show up to 9.65× compression ratio and up to 4.79× higher end-to-end AllReduce throughput than baseline NCCL on representative scientific and training tensors.