Accelerating Lossy and Lossless Compression on Emerging BlueField DPU Architectures
Authors:
- 27 May 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 373-385
Abstract
Data compression has become a crucial technique in addressing performance bottlenecks caused by increasing data volumes in High-Performance Computing (HPC), Big Data, and Deep Learning (DL). Despite its potential to boost system performance, recent studies have identified significant challenges with existing compression methods, mainly due to their high computational demands amidst continuously growing data sizes. Concurrently, the advent of Data Processing Units (DPUs), equipped with programmable System-on-Chip (SoC) and specialized compression accelerators, offers a promising opportunity to alter the landscape of data compression. This paper explores the complexities and potential of leveraging NVIDIA BlueField DPUs to accelerate lossy and lossless compression. Towards this, we introduce PEDAL, an innovative library that leverages the hardware capabilities of DPUs to unify and optimize data compression designs. Moreover, we seamlessly co-design PEDAL with the popular MPICH MPI library, demonstrating up to 101x speedup in compression time and 88x decrease in communication latency. Drawing on these achievements, we share our experience with various research communities about accelerating data compression on DPUs in communication-oriented HPC scenarios.Keywords
This publication has 32 references indexed in Scilit:
- Battle of the BlueFields: An In-Depth Comparison of the BlueField-2 and BlueField-3 SmartNICs2023
- Characterizing Lossy and Lossless Compression on Emerging BlueField DPU Architectures2023
- Runway: In-transit Data Compression on Heterogeneous HPC Systems2023
- Accelerating MPI All-to-All Communication with Online Compression on Modern GPU Clusters2022
- SDRBench: Scientific Data Reduction Benchmark for Lossy Compressors2020
- Improving semantic compression specification in large relational databaseIET Software, 2016
- Triple-H: A Hybrid Approach to Accelerate HDFS on HPC Clusters with Heterogeneous Storage Architecture2015
- Accelerating Spark with RDMA for Big Data Processing: Early Experiences2014
- CoMPI: Enhancing MPI Based Applications Performance and Scalability Using Run-Time Compression2009
- A data mining approach to database compressionInformation Systems Frontiers, 2006