POSTER: Optimizing Collective Communications with Error-bounded Lossy Compression for GPU Clusters
Open Access
- 20 February 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 454-456
Abstract
GPU-aware collective communication has become a major bottleneck for modern computing platforms as GPU computing power rapidly rises. To address this issue, traditional approaches integrate lossy compression directly into GPU-aware collectives, which still suffer from serious issues such as underutilized GPU devices and uncontrolled data distortion. In this paper, we propose GPU-LCC, a general framework that designs and optimizes GPU-aware, compression-enabled collectives with well-controlled error propagation. To validate our framework, we evaluate the performance on up to 64 NVIDIA A100 GPUs with real-world applications and datasets. Experimental results demonstrate that our GPU-LCC-accelerated collective computation (Allreduce), can outperform NCCL as well as Cray MPI by up to 3.4× and 18.7×, respectively. Furthermore, our accuracy evaluation with an image-stacking application confirms the high reconstructed data quality of our accuracy-aware framework.Keywords
Funding Information
- NSF (National Science Foundation) (OAC-2003709, OAC-2104023)
- DOE U.S. Department of Energy (Advanced Scientific Computing Research (ASCR))
This publication has 13 references indexed in Scilit:
- Accelerating MPI Collectives with Process-in-Process-based Multi-object Techniques2023
- Designing High-Performance MPI Libraries with On-the-fly Compression for Modern GPU Clusters2021
- Significantly Improving Lossy Compression for HPC Datasets with Second-Order Prediction and Parameter Optimization2020
- GeoDRIVE - a high performance computing flexible platform for seismic applicationsFirst Break, 2020
- Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization2017
- Fast Error-Bounded Lossy HPC Data Compression with SZ2016
- Fixed-Rate Compressed Floating-Point ArraysIEEE Transactions on Visualization and Computer Graphics, 2014
- Bandwidth optimal all-reduce algorithms for clusters of workstationsJournal of Parallel and Distributed Computing, 2009
- Optimization of MPI collective communication on BlueGene/L systems2005
- Optimization of Collective Communication Operations in MPICHThe International Journal of High Performance Computing Applications, 2005