SDRBench: Scientific Data Reduction Benchmark for Lossy Compressors
Authors:
Top Cited Papers
- 10 December 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2716-2724
Abstract
Efficient error-controlled lossy compressors are becoming critical to the success of today's large-scale scientific applications because of the ever-increasing volume of data produced by the applications. In the past decade, many lossless and lossy compressors have been developed with distinct design principles for different scientific datasets in largely diverse scientific domains. In order to support researchers and users assessing and comparing compressors in a fair and convenient way, we establish a standard compression assessment benchmark - Scientific Data Reduction Benchmark (SDRBench) 1 . SDRBench contains a vast variety of real-world scientific datasets across different domains, summarizes several critical compression quality evaluation metrics, and integrates many state-of-the-art lossy and lossless compressors. We demonstrate evaluation results using SDRBench and summarize six valuable takeaways that are helpful to the in-depth understanding of lossy compressors.Keywords
All Related Versions
Funding Information
- National Science Foundation
This publication has 22 references indexed in Scilit:
- Z-checker: A framework for assessing lossy compression of scientific dataThe International Journal of High Performance Computing Applications, 2017
- Exploration of Pattern-Matching Techniques for Lossy Compression on Cosmology Simulation Data Sets2017
- Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization2017
- The Near-Real-Time SCALE-LETKF System: A Case of the September 2015 Kanto-Tohoku Heavy RainfallSOLA, 2017
- HACCCommunications of the ACM, 2016
- Fast Error-Bounded Lossy HPC Data Compression with SZ2016
- Fixed-Rate Compressed Floating-Point ArraysIEEE Transactions on Visualization and Computer Graphics, 2014
- Mean squared error: Love it or leave it? A new look at Signal Fidelity MeasuresIEEE Signal Processing Magazine, 2009
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- GZIP file format specification version 4.31996