Xiaodong Yu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Xiaodong Yu has a total of 68 co-authors. Here are the top co-authors by number of shared publications:
- Sheng Di
Argonne National Laboratory, Lemont, IL, USA
10 shared publications - Franck Cappello
Argonne National Laboratory, Lemont, USA
8 shared publications - Dingwen Tao
Chinese Academy of Sciences
6 shared publications - Jiannan Tian
Oakland University, Rochester, USA
5 shared publications - Yafan Huang
University of Iowa, Iowa City, IA, USA
4 shared publications - Baixi Sun
Indiana University, Bloomington, IN, USA
3 shared publications
Latest Publications
- Open Access
- 13 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 577-578
NCCLZ: Compression-Enabled GPU Collectives with Decoupled Quantization and Entropy Coding
Large multi-node GPU jobs often become limited by the number of bytes injected into the inter-node network path. Compression can reduce this traffic, but existing compression-assisted collective libraries either sit outside NCCL’s GPU-centric execution model or mainly rely on quantization alone. We present NCCLZ, a compression-enabled extension to NCCL that decouples lossy quantization from lossless entropy coding. NCCLZ performs approximation once before communication by converting tensors into integer symbols, and then applies lossless byte reduction inside NCCL’s communication pipeline. This placement keeps numerical approximation explicit, avoids repeated lossy recompression during reductions, and preserves NCCL’s native pipelined execution model. NCCLZ combines a runtime selector with slot-aware framing and pipelined staging to choose among raw transmission, fixed-length packing, and GPU Huffman coding. Initial results on Polaris show up to 9.65× compression ratio and up to 4.79× higher end-to-end AllReduce throughput than baseline NCCL on representative scientific and training tensors.
- 13 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 577-578
Open AccessNCCLZ: Compression-Enabled GPU Collectives with Decoupled Quantization and Entropy Coding
Large multi-node GPU jobs often become limited by the number of bytes injected into the inter-node network path. Compression can reduce this traffic, but existing compression-assisted collective libraries either sit outside NCCL’s GPU-centric execution model or mainly rely on quantization alone. We present NCCLZ, a compression-enabled extension to NCCL that decouples lossy quantization from lossless entropy coding. NCCLZ performs approximation once before communication by converting tensors into integer symbols, and then applies lossless byte reduction inside NCCL’s communication pipeline. This placement keeps numerical approximation explicit, avoids repeated lossy recompression during reductions, and preserves NCCL’s native pipelined execution model. NCCLZ combines a runtime selector with slot-aware framing and pipelined staging to choose among raw transmission, fixed-length packing, and GPU Huffman coding. Initial results on Polaris show up to 9.65× compression ratio and up to 4.79× higher end-to-end AllReduce throughput than baseline NCCL on representative scientific and training tensors.
- Open Access
- 25 January 2026
- Published by Association for Computing Machinery (ACM)
- p. 274-283
DeepEBC: Compressing the Pre-Trained LLMs with Error-Bounded Lossy Compression
- 25 January 2026
- Published by Association for Computing Machinery (ACM)
- p. 274-283
Open AccessDeepEBC: Compressing the Pre-Trained LLMs with Error-Bounded Lossy Compression
- Open Access
- 3 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 312-323
EvaMAE: How Helpful Are DEM Data in Enhancing Geo-Foundation Models for Earth Imagery?
- 3 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 312-323
Open AccessEvaMAE: How Helpful Are DEM Data in Enhancing Geo-Foundation Models for Earth Imagery?
- Top Cited PapersOpen Access
- 11 June 2025
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 57  (11) , 1-38
A Survey on Error-Bounded Lossy Compression for Scientific Datasets
Error-bounded lossy compression has been effective in significantly reducing the data storage/transfer burden while preserving the reconstructed data fidelity very well. Many error-bounded lossy compressors have been developed for a wide range of parallel and distributed use cases for years. They are designed with distinct compression models and principles, such that each of them features particular pros and cons. In this article, we provide a comprehensive survey of emerging error-bounded lossy compression techniques. The key contribution is fourfold. (1) We summarize a novel taxonomy of lossy compression into six classic models. (2) We provide a comprehensive survey of 10 commonly used compression components/modules. (3) We summarized pros and cons of 47 state-of-the-art lossy compressors and present how state-of-the-art compressors are designed based on different compression techniques. (4) We discuss how customized compressors are designed for specific scientific applications and use-cases. We believe this survey is useful to multiple communities including scientific applications, high-performance computing, lossy compression, and big data.
- 11 June 2025
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 57  (11) , 1-38
Top Cited PapersOpen AccessA Survey on Error-Bounded Lossy Compression for Scientific Datasets
Error-bounded lossy compression has been effective in significantly reducing the data storage/transfer burden while preserving the reconstructed data fidelity very well. Many error-bounded lossy compressors have been developed for a wide range of parallel and distributed use cases for years. They are designed with distinct compression models and principles, such that each of them features particular pros and cons. In this article, we provide a comprehensive survey of emerging error-bounded lossy compression techniques. The key contribution is fourfold. (1) We summarize a novel taxonomy of lossy compression into six classic models. (2) We provide a comprehensive survey of 10 commonly used compression components/modules. (3) We summarized pros and cons of 47 state-of-the-art lossy compressors and present how state-of-the-art compressors are designed based on different compression techniques. (4) We discuss how customized compressors are designed for specific scientific applications and use-cases. We believe this survey is useful to multiple communities including scientific applications, high-performance computing, lossy compression, and big data.
- Open Access
- 28 February 2025
- Published by Association for Computing Machinery (ACM)
- p. 212-224
COMPSO: Optimizing Gradient Compression for Distributed Training with Second-Order Optimizers
Second-order optimization methods have been developed to enhance convergence and generalization in deep neural network (DNN) training compared to first-order methods like Stochastic Gradient Descent (SGD). However, these methods face challenges in distributed settings due to high communication overhead. Gradient compression, a technique commonly used to accelerate communication for first-order approaches, often results in low communication reduction ratios, decreased model accuracy, and/or high compression overhead when applied to second-order methods. To address these limitations, we introduce a novel gradient compression method for second-order optimizers called COMPSO. This method effectively reduces communication costs while preserving the advantages of second-order optimization. COMPSO employs stochastic rounding to maintain accuracy and filters out minor gradients to improve compression ratios. Additionally, we develop GPU optimizations to minimize compression overhead and performance modeling to ensure end-to-end performance gains across various systems. Evaluation of COMPSO on different DNN models shows that it achieves a compression ratio of 22.1×, reduces communication time by 14.2×, and improves overall performance by 1.9×, all without any drop in model accuracy.
- 28 February 2025
- Published by Association for Computing Machinery (ACM)
- p. 212-224
Open AccessCOMPSO: Optimizing Gradient Compression for Distributed Training with Second-Order Optimizers
Second-order optimization methods have been developed to enhance convergence and generalization in deep neural network (DNN) training compared to first-order methods like Stochastic Gradient Descent (SGD). However, these methods face challenges in distributed settings due to high communication overhead. Gradient compression, a technique commonly used to accelerate communication for first-order approaches, often results in low communication reduction ratios, decreased model accuracy, and/or high compression overhead when applied to second-order methods. To address these limitations, we introduce a novel gradient compression method for second-order optimizers called COMPSO. This method effectively reduces communication costs while preserving the advantages of second-order optimization. COMPSO employs stochastic rounding to maintain accuracy and filters out minor gradients to improve compression ratios. Additionally, we develop GPU optimizations to minimize compression overhead and performance modeling to ensure end-to-end performance gains across various systems. Evaluation of COMPSO on different DNN models shows that it achieves a compression ratio of 22.1×, reduces communication time by 14.2×, and improves overall performance by 1.9×, all without any drop in model accuracy.
- Open Access
- 3 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 109-121
A Portable, Fast, DCT-based Compressor for AI Accelerators
Lossy compression can be an effective tool in AI training and inference to reduce memory requirements, storage footprint, and in some cases, execution time. With the rise of novel architectures designed to accelerate AI workloads, compression can continue to serve these purposes, but must be adapted to the new accelerators. Due to programmability and architectural differences, existing lossy compressors cannot be directly ported to and are not optimized for any AI accelerator, thus requiring new compression designs. In this paper, we propose a novel, portable, DCT-based lossy compressor that can be used across a variety of AI accelerators. More specifically, we make the following contributions: 1) We propose a DCT-based lossy compressor design for training data that uses operators supported across four state-of-the-art AI accelerators: Cerebras CS-2, SambaNova SN30, Groq GroqChip, and Graphcore IPU. 2) We design two optimization techniques to allow for higher resolution compressed data on certain platforms and improved compression ratio on the IPU. 3) We evaluate our compressor's ability to preserve accuracy on four benchmarks, three of which are AI for science benchmarks going beyond image classification. Our experiments show that accuracy degradation can be limited to 3% or less, and sometimes, compression improves accuracy. 4) We study compression/decompression time as a function of resolution and batch size, finding that our compressor can achieve throughputs on the scale of tens of GB/s, depending on the platform.
- 3 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 109-121
Open AccessA Portable, Fast, DCT-based Compressor for AI Accelerators
Lossy compression can be an effective tool in AI training and inference to reduce memory requirements, storage footprint, and in some cases, execution time. With the rise of novel architectures designed to accelerate AI workloads, compression can continue to serve these purposes, but must be adapted to the new accelerators. Due to programmability and architectural differences, existing lossy compressors cannot be directly ported to and are not optimized for any AI accelerator, thus requiring new compression designs. In this paper, we propose a novel, portable, DCT-based lossy compressor that can be used across a variety of AI accelerators. More specifically, we make the following contributions: 1) We propose a DCT-based lossy compressor design for training data that uses operators supported across four state-of-the-art AI accelerators: Cerebras CS-2, SambaNova SN30, Groq GroqChip, and Graphcore IPU. 2) We design two optimization techniques to allow for higher resolution compressed data on certain platforms and improved compression ratio on the IPU. 3) We evaluate our compressor's ability to preserve accuracy on four benchmarks, three of which are AI for science benchmarks going beyond image classification. Our experiments show that accuracy degradation can be limited to 3% or less, and sometimes, compression improves accuracy. 4) We study compression/decompression time as a function of resolution and batch size, finding that our compressor can achieve throughputs on the scale of tens of GB/s, depending on the platform.
- Open Access
- 3 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 309-321
CereSZ: Enabling and Scaling Error-bounded Lossy Compression on Cerebras CS-2
Today's scientific applications running on supercomputers produce large volumes of data, leading to critical data storage and communication challenges. To tackle the challenges, error-bounded lossy compression is commonly adopted since it can reduce data size drastically within a user-defined error threshold. Previous work has shown that compression techniques can significantly reduce the storage and I/O overhead while retaining good data quality. However, the existing compressors are mainly designed for CPU and GPU. As new AI chips are being incorporated into supercomputers and increasingly used for accelerating scientific computing, there is a growing demand for efficient data compression on the new architecture. In this paper, we propose an efficient lossy compressor, CereSZ, based on the Cerebras CS-2 system. The compression algorithm is mapped onto Cerebras using both data parallelism and pipeline parallelism. In order to achieve a balanced workload on each processing unit, we propose an algorithm to evenly distribute the pipeline stages. Our experiments with six scientific datasets demonstrate that CereSZ can achieve a throughput from 227.93 GB/s to 773.8 GB/s, 2.43x to 10.98x faster than existing GPU compressors.
- 3 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 309-321
Open AccessCereSZ: Enabling and Scaling Error-bounded Lossy Compression on Cerebras CS-2
Today's scientific applications running on supercomputers produce large volumes of data, leading to critical data storage and communication challenges. To tackle the challenges, error-bounded lossy compression is commonly adopted since it can reduce data size drastically within a user-defined error threshold. Previous work has shown that compression techniques can significantly reduce the storage and I/O overhead while retaining good data quality. However, the existing compressors are mainly designed for CPU and GPU. As new AI chips are being incorporated into supercomputers and increasingly used for accelerating scientific computing, there is a growing demand for efficient data compression on the new architecture. In this paper, we propose an efficient lossy compressor, CereSZ, based on the Cerebras CS-2 system. The compression algorithm is mapped onto Cerebras using both data parallelism and pipeline parallelism. In order to achieve a balanced workload on each processing unit, we propose an algorithm to evenly distribute the pipeline stages. Our experiments with six scientific datasets demonstrate that CereSZ can achieve a throughput from 227.93 GB/s to 773.8 GB/s, 2.43x to 10.98x faster than existing GPU compressors.
- Open Access
- 30 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 437-448
gZCCL: Compression-Accelerated Collective Communication Framework for GPU Clusters
GPU-aware collective communication has become a major bottleneck for modern computing platforms as GPU computing power rapidly rises. A traditional approach is to directly integrate lossy compression into GPU-aware collectives, which can lead to serious performance issues such as underutilized GPU devices and uncontrolled data distortion. In order to address these issues, in this paper, we propose gZCCL, a first-ever general framework that designs and optimizes GPU-aware, compression-enabled collectives with an accuracy-aware design to control error propagation. To validate our framework, we evaluate the performance on up to 512 NVIDIA A100 GPUs with real-world applications and datasets. Experimental results demonstrate that our gZCCL-accelerated collectives, including both collective computation (Allreduce) and collective data movement (Scatter), can outperform NCCL as well as Cray MPI by up to 4.5 × and 28.7 ×, respectively. Furthermore, our accuracy evaluation with an image-stacking application confirms the high reconstructed data quality of our accuracy-aware framework.
- 30 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 437-448
Open AccessgZCCL: Compression-Accelerated Collective Communication Framework for GPU Clusters
GPU-aware collective communication has become a major bottleneck for modern computing platforms as GPU computing power rapidly rises. A traditional approach is to directly integrate lossy compression into GPU-aware collectives, which can lead to serious performance issues such as underutilized GPU devices and uncontrolled data distortion. In order to address these issues, in this paper, we propose gZCCL, a first-ever general framework that designs and optimizes GPU-aware, compression-enabled collectives with an accuracy-aware design to control error propagation. To validate our framework, we evaluate the performance on up to 512 NVIDIA A100 GPUs with real-world applications and datasets. Experimental results demonstrate that our gZCCL-accelerated collectives, including both collective computation (Allreduce) and collective data movement (Scatter), can outperform NCCL as well as Cray MPI by up to 4.5 × and 28.7 ×, respectively. Furthermore, our accuracy evaluation with an image-stacking application confirms the high reconstructed data quality of our accuracy-aware framework.
- Open Access
- 20 February 2024
- Published by Association for Computing Machinery (ACM)
- p. 454-456
POSTER: Optimizing Collective Communications with Error-bounded Lossy Compression for GPU Clusters
GPU-aware collective communication has become a major bottleneck for modern computing platforms as GPU computing power rapidly rises. To address this issue, traditional approaches integrate lossy compression directly into GPU-aware collectives, which still suffer from serious issues such as underutilized GPU devices and uncontrolled data distortion. In this paper, we propose GPU-LCC, a general framework that designs and optimizes GPU-aware, compression-enabled collectives with well-controlled error propagation. To validate our framework, we evaluate the performance on up to 64 NVIDIA A100 GPUs with real-world applications and datasets. Experimental results demonstrate that our GPU-LCC-accelerated collective computation (Allreduce), can outperform NCCL as well as Cray MPI by up to 3.4× and 18.7×, respectively. Furthermore, our accuracy evaluation with an image-stacking application confirms the high reconstructed data quality of our accuracy-aware framework.
- 20 February 2024
- Published by Association for Computing Machinery (ACM)
- p. 454-456
Open AccessPOSTER: Optimizing Collective Communications with Error-bounded Lossy Compression for GPU Clusters
GPU-aware collective communication has become a major bottleneck for modern computing platforms as GPU computing power rapidly rises. To address this issue, traditional approaches integrate lossy compression directly into GPU-aware collectives, which still suffer from serious issues such as underutilized GPU devices and uncontrolled data distortion. In this paper, we propose GPU-LCC, a general framework that designs and optimizes GPU-aware, compression-enabled collectives with well-controlled error propagation. To validate our framework, we evaluate the performance on up to 64 NVIDIA A100 GPUs with real-world applications and datasets. Experimental results demonstrate that our GPU-LCC-accelerated collective computation (Allreduce), can outperform NCCL as well as Cray MPI by up to 3.4× and 18.7×, respectively. Furthermore, our accuracy evaluation with an image-stacking application confirms the high reconstructed data quality of our accuracy-aware framework.
- Open Access
- 12 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 1759-1766
Benchmarking and In-depth Performance Study of Large Language Models on Habana Gaudi Processors
Transformer models have achieved remarkable success in various machine learning tasks but suffer from high computational complexity and resource requirements. The quadratic complexity of the self-attention mechanism further exacerbates these challenges when dealing with long sequences and large datasets. Specialized AI hardware accelerators, such as the Habana GAUDI architecture, offer a promising solution to tackle these issues. GAUDI features a Matrix Multiplication Engine (MME) and a cluster of fully programmable Tensor Processing Cores (TPC). This paper explores the untapped potential of using GAUDI processors to accelerate Transformer-based models, addressing key challenges in the process. Firstly, we provide a comprehensive performance comparison between the MME and TPC components, illuminating their relative strengths and weaknesses. Secondly, we explore strategies to optimize MME and TPC utilization, offering practical insights to enhance computational efficiency. Thirdly, we evaluate the performance of Transformers on GAUDI, particularly in handling long sequences and uncovering performance bottlenecks. Lastly, we evaluate the end-to-end performance of two Transformer-based large language models (LLM) on GAUDI. The contributions of this work encompass practical insights for practitioners and researchers alike. We delve into GAUDI’s capabilities for Transformers through systematic profiling, analysis, and optimization exploration. Our study bridges a research gap and offers a roadmap for optimizing Transformer-based model training on the GAUDI architecture.
- 12 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 1759-1766
Open AccessBenchmarking and In-depth Performance Study of Large Language Models on Habana Gaudi Processors
Transformer models have achieved remarkable success in various machine learning tasks but suffer from high computational complexity and resource requirements. The quadratic complexity of the self-attention mechanism further exacerbates these challenges when dealing with long sequences and large datasets. Specialized AI hardware accelerators, such as the Habana GAUDI architecture, offer a promising solution to tackle these issues. GAUDI features a Matrix Multiplication Engine (MME) and a cluster of fully programmable Tensor Processing Cores (TPC). This paper explores the untapped potential of using GAUDI processors to accelerate Transformer-based models, addressing key challenges in the process. Firstly, we provide a comprehensive performance comparison between the MME and TPC components, illuminating their relative strengths and weaknesses. Secondly, we explore strategies to optimize MME and TPC utilization, offering practical insights to enhance computational efficiency. Thirdly, we evaluate the performance of Transformers on GAUDI, particularly in handling long sequences and uncovering performance bottlenecks. Lastly, we evaluate the end-to-end performance of two Transformer-based large language models (LLM) on GAUDI. The contributions of this work encompass practical insights for practitioners and researchers alike. We delve into GAUDI’s capabilities for Transformers through systematic profiling, analysis, and optimization exploration. Our study bridges a research gap and offers a roadmap for optimizing Transformer-based model training on the GAUDI architecture.