Scientific Error-bounded Lossy Compression with Super-resolution Neural Networks
Authors:
- 15 December 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 229-236
Abstract
The fast growth of computational power and scales of modern super-computing systems have raised great challenges for the management of exascale scientific data. To maintain the usability of scientific data, error-bound lossy compression is proposed and developed as an essential technique for the size reduction of scientific data with constrained data distortion. Among the diverse datasets generated by various scientific simulations, certain datasets cannot be effectively compressed by existing error-bounded lossy compressors with traditional techniques. The recent success of Artificial Intelligence has inspired several researchers to integrate neural networks into error-bounded lossy compressors. However, those works still suffer from limited compression ratios and/or extremely low efficiencies. To address those issues and improve the compression on the hard-to-compress datasets, in this paper, we propose SRNNSZ, which is a deep learning-based scientific error-bounded lossy compressor leveraging the hierarchical data grid expansion paradigm implemented by super-resolution neural networks. SRNN-SZ applies the most advanced super-resolution network HAT for its compression, which is free of time-costing per-data training. In experiments compared with various state-of-the-art compressors, SRNN-SZ achieves up to 75% compression ratio improvements under the same error bound and up to 80% compression ratio improvements under the same PSNR than the second-best compressor.Keywords
Funding Information
- U.S. Department of Energy
- Office of Science
- Advanced Scientific Computing Research
- National Science Foundation
This publication has 29 references indexed in Scilit:
- Using Neural Networks for Two Dimensional Scientific Data Compression2021
- Exploring Autoencoder-based Error-bounded Compression for Scientific Data2021
- SDRBench: Scientific Data Reduction Benchmark for Lossy Compressors2020
- Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets2018
- Multilevel techniques for compression and reduction of scientific data—the univariate caseComputing and Visualization in Science, 2018
- Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network2018
- Deep Residual Learning for Image Recognition2016
- Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network2016
- Learning a Deep Convolutional Network for Image Super-Resolution2014
- A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulenceJournal of Turbulence, 2008