Shaomeng Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Shaomeng Li has a total of 62 co-authors. Here are the top co-authors by number of shared publications:
- Hank Childs
University of Oregon, Eugene, OR, USA
4 shared publications - John Clyne
National Center for Atmospheric Research
3 shared publications - Allison H. Baker
NSF National Center for Atmospheric Research, Boulder, CO, USA
2 shared publications Abhishek Nair
1 shared publicationAl Dunstan
1 shared publicationAlejandro Chacon
1 shared publication
Latest Publications
- Open Access
- 13 March 2026
- Published by Copernicus GmbH
Application of advanced lossy compression in the NetCDF ecosystem for CONUS404 data
Many geoscientific datasets, such as those produced by climate and weather models, are stored in the NetCDF file format. These datasets are typically very large and often strain institutional data storage resources. While lossy compression methods for scientific data have become more studied and adopted in recent years, most advanced lossy approaches do not work easily and/or transparently with NetCDF files. For example, they may require a file format conversion or they may not work correctly with “missing values” or “fill values” that are often present in model outputs. While lossy quantization approaches such at BitRound and Granular BitRound have built-in support by NetCDF and are quite easy to use, such approaches are generally not able to reduce the data size as much as more advanced compressors (for a fixed error metric), like SPERR, ZFP, or SZ3.We are particularly interested in reducing the data size of the CONUS404 dataset. CONUS404 is a publicly available unique high-resolution hydro-climate dataset produced by Weather Research and Forecasting (WRF) Model simulations that cover the CONtiguous United States (CONUS) for 40 years at 4-km resolution (a collaboration between NSF National Center for Atmospheric Research the U.S. Geological Survey Water Mission Area). Here, we investigate one advanced lossy compressor, SPERR [1], together with its plugin for NetCDF files, H5Z-SPERR [2], in a Python-based workflow to compress and analyze CONUS404 data. SPERR is attractive due to its support for quality control in terms of both maximum point-wise error (PWE) and peak signal-to-noise ratio (PSNR), enabling easy experimenting of storage-quality tradeoffs. Further, given a target quality metric, previous work has shown that SPERR likely produces the smallest compressed file size compared to other advanced compressors. It leverages the HDF5 dynamic plugin mechanism to enable users to stay in the NetCDF ecosystem with minimal to no change to existing analysis workflows, whenever a typical NetCDF file is able to be read. And, importantly for our work, the SPERR plugin supports efficient masking of “missing values,” which are common to climate and weather model output. The support for missing values enables compression on many variables which are not naturally handled by other advanced compressors that rely on HDF5 plugins. Further, because H5Z-SPERR directly handles missing values, they can be stored in a much more compact format (and are restored during decompression), further improving compression efficiency. (Note that built-in NetCDF quantization approaches can work with missing values.) Our experimentation demonstrates the benefit of enabling advanced lossy (de)compression in the NetCDF ecosystem: adoption friction is kept at the minimum with little change to workflows, while storage requirements are greatly reduced. [1] https://github.com/NCAR/SPERR[2] https://github.com/NCAR/H5Z-SPERR
- 13 March 2026
- Published by Copernicus GmbH
Open AccessApplication of advanced lossy compression in the NetCDF ecosystem for CONUS404 data
Many geoscientific datasets, such as those produced by climate and weather models, are stored in the NetCDF file format. These datasets are typically very large and often strain institutional data storage resources. While lossy compression methods for scientific data have become more studied and adopted in recent years, most advanced lossy approaches do not work easily and/or transparently with NetCDF files. For example, they may require a file format conversion or they may not work correctly with “missing values” or “fill values” that are often present in model outputs. While lossy quantization approaches such at BitRound and Granular BitRound have built-in support by NetCDF and are quite easy to use, such approaches are generally not able to reduce the data size as much as more advanced compressors (for a fixed error metric), like SPERR, ZFP, or SZ3.We are particularly interested in reducing the data size of the CONUS404 dataset. CONUS404 is a publicly available unique high-resolution hydro-climate dataset produced by Weather Research and Forecasting (WRF) Model simulations that cover the CONtiguous United States (CONUS) for 40 years at 4-km resolution (a collaboration between NSF National Center for Atmospheric Research the U.S. Geological Survey Water Mission Area). Here, we investigate one advanced lossy compressor, SPERR [1], together with its plugin for NetCDF files, H5Z-SPERR [2], in a Python-based workflow to compress and analyze CONUS404 data. SPERR is attractive due to its support for quality control in terms of both maximum point-wise error (PWE) and peak signal-to-noise ratio (PSNR), enabling easy experimenting of storage-quality tradeoffs. Further, given a target quality metric, previous work has shown that SPERR likely produces the smallest compressed file size compared to other advanced compressors. It leverages the HDF5 dynamic plugin mechanism to enable users to stay in the NetCDF ecosystem with minimal to no change to existing analysis workflows, whenever a typical NetCDF file is able to be read. And, importantly for our work, the SPERR plugin supports efficient masking of “missing values,” which are common to climate and weather model output. The support for missing values enables compression on many variables which are not naturally handled by other advanced compressors that rely on HDF5 plugins. Further, because H5Z-SPERR directly handles missing values, they can be stored in a much more compact format (and are restored during decompression), further improving compression efficiency. (Note that built-in NetCDF quantization approaches can work with missing values.) Our experimentation demonstrates the benefit of enabling advanced lossy (de)compression in the NetCDF ecosystem: adoption friction is kept at the minimum with little change to workflows, while storage requirements are greatly reduced. [1] https://github.com/NCAR/SPERR[2] https://github.com/NCAR/H5Z-SPERR
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1966-1979
What to Support When You’re Compressing
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1966-1979
Open AccessWhat to Support When You’re Compressing
- Open Access
Parabricks: GPU Accelerated Universal Pan-Instrument Genomics Analysis Software Suite
Next-generation sequencing (NGS) has transformed genomics, enabling breakthroughs in biotechnology, healthcare, and pharmaceuticals. However, exponential data growth-outpacing Moore’s Law-presents critical computational challenges for secondary analysis. We introduce Parabricks, a freely accessible, GPU-accelerated software suite supporting diverse workflows, including wholegenome, exome, transcriptome, and methylation analysis. Compatible across major short read and long read sequencing platforms, it reduces processing time from hours to minutes while maintaining concordance with established standards, this acceleration translates to significant cost savings. Deployable on-premises or through cloud platforms, Parabricks ensures scalable, genomic processing. Integrated AI-driven workflows and deep learning models further enhance precision, offering a universal solution for high-throughput genomic research and clinical diagnostics.
Open AccessParabricks: GPU Accelerated Universal Pan-Instrument Genomics Analysis Software Suite
Next-generation sequencing (NGS) has transformed genomics, enabling breakthroughs in biotechnology, healthcare, and pharmaceuticals. However, exponential data growth-outpacing Moore’s Law-presents critical computational challenges for secondary analysis. We introduce Parabricks, a freely accessible, GPU-accelerated software suite supporting diverse workflows, including wholegenome, exome, transcriptome, and methylation analysis. Compatible across major short read and long read sequencing platforms, it reduces processing time from hours to minutes while maintaining concordance with established standards, this acceleration translates to significant cost savings. Deployable on-premises or through cloud platforms, Parabricks ensures scalable, genomic processing. Integrated AI-driven workflows and deep learning models further enhance precision, offering a universal solution for high-throughput genomic research and clinical diagnostics.
- 5 May 2022
- Published by Springer Nature
- p. 37-59
In Situ Wavelet Compression on Supercomputers for Post Hoc Exploration
- 5 May 2022
- Published by Springer Nature
- p. 37-59
In Situ Wavelet Compression on Supercomputers for Post Hoc Exploration
- Open Access
- 26 January 2020
- Published by Society for Imaging Science & Technology in Electronic Imaging
- Vol. 32 (1) , 376-1
HashFight: A Platform-Portable Hash Table for Multi-Core and Many-Core Architectures
Fast track article for IS&T International Symposium on Electronic Imaging 2020: Visualization and Data Analysis proceedings.
- 26 January 2020
- Published by Society for Imaging Science & Technology in Electronic Imaging
- Vol. 32 (1) , 376-1
Open AccessHashFight: A Platform-Portable Hash Table for Multi-Core and Many-Core Architectures
Fast track article for IS&T International Symposium on Electronic Imaging 2020: Visualization and Data Analysis proceedings.
- Open Access
- 25 August 2019
- Published by MDPI AG in Atmosphere
- Vol. 10 (9) , 488
VAPOR: A Visualization Package Tailored to Analyze Simulation Data in Earth System Science
Visualization is an essential tool for analysis of data and communication of findings in the sciences, and the Earth System Sciences (ESS) are no exception. However, within ESS, specialized visualization requirements and data models, particularly for those data arising from numerical models, often make general purpose visualization packages difficult, if not impossible, to use effectively. This paper presents VAPOR: a domain-specific visualization package that targets the specialized needs of ESS modelers, particularly those working in research settings where highly-interactive exploratory visualization is beneficial. We specifically describe VAPOR’s ability to handle ESS simulation data from a wide variety of numerical models, as well as a multi-resolution representation that enables interactive visualization on very large data while using only commodity computing resources. We also describe VAPOR’s visualization capabilities, paying particular attention to features for geo-referenced data and advanced rendering algorithms suitable for time-varying, 3D data. Finally, we illustrate VAPOR’s utility in the study of a numerically- simulated tornado. Our results demonstrate both ease-of-use and the rich capabilities of VAPOR in such a use case.
- 25 August 2019
- Published by MDPI AG in Atmosphere
- Vol. 10 (9) , 488
Open AccessVAPOR: A Visualization Package Tailored to Analyze Simulation Data in Earth System Science
Visualization is an essential tool for analysis of data and communication of findings in the sciences, and the Earth System Sciences (ESS) are no exception. However, within ESS, specialized visualization requirements and data models, particularly for those data arising from numerical models, often make general purpose visualization packages difficult, if not impossible, to use effectively. This paper presents VAPOR: a domain-specific visualization package that targets the specialized needs of ESS modelers, particularly those working in research settings where highly-interactive exploratory visualization is beneficial. We specifically describe VAPOR’s ability to handle ESS simulation data from a wide variety of numerical models, as well as a multi-resolution representation that enables interactive visualization on very large data while using only commodity computing resources. We also describe VAPOR’s visualization capabilities, paying particular attention to features for geo-referenced data and advanced rendering algorithms suitable for time-varying, 3D data. Finally, we illustrate VAPOR’s utility in the study of a numerically- simulated tornado. Our results demonstrate both ease-of-use and the rich capabilities of VAPOR in such a use case.
- Open Access
- 25 July 2019
- Published by MDPI AG in Preprints.org
VAPOR: A Visualization Package Tailored to Analyze Simulation Data in Earth System Science
Visualization is an essential tool for analysis of data and communication of findings in the sciences, and the Earth System Science (ESS) are no exception. However, within ESS specialized visualization requirements and data models --- particularly for those data arising from numerical models --- often make general-purpose visualization packages difficult, if not impossible, to effectively use. This paper presents VAPOR: a domain-specific visualization package that targets the specialized needs of ESS modelers, particularly those working in research settings where highly interactive exploratory visualization is beneficial. We specifically describe VAPOR’s ability to handle ESS simulation data from a wide variety of numerical models, as well as a multi-resolution representation that enables interactive visualization on very large data while using only commodity computing resources. We also describe VAPOR’s visualization capabilities, paying particular attention to features for geo-referenced data and advanced rendering algorithms suitable for time-varying, 3D data. Finally, we illustrate VAPOR's utility in the study of a numerically simulated tornado. Our results demonstrate both ease-of-use and the rich capabilities of VAPOR in such a use case.
- 25 July 2019
- Published by MDPI AG in Preprints.org
Open AccessVAPOR: A Visualization Package Tailored to Analyze Simulation Data in Earth System Science
Visualization is an essential tool for analysis of data and communication of findings in the sciences, and the Earth System Science (ESS) are no exception. However, within ESS specialized visualization requirements and data models --- particularly for those data arising from numerical models --- often make general-purpose visualization packages difficult, if not impossible, to effectively use. This paper presents VAPOR: a domain-specific visualization package that targets the specialized needs of ESS modelers, particularly those working in research settings where highly interactive exploratory visualization is beneficial. We specifically describe VAPOR’s ability to handle ESS simulation data from a wide variety of numerical models, as well as a multi-resolution representation that enables interactive visualization on very large data while using only commodity computing resources. We also describe VAPOR’s visualization capabilities, paying particular attention to features for geo-referenced data and advanced rendering algorithms suitable for time-varying, 3D data. Finally, we illustrate VAPOR's utility in the study of a numerically simulated tornado. Our results demonstrate both ease-of-use and the rich capabilities of VAPOR in such a use case.
- 30 March 2018
- Published by Wiley in Computer Graphics Forum
- Vol. 37 (6) , 422-447
Data Reduction Techniques for Simulation, Visualization and Data Analysis
Data reduction is increasingly being applied to scientific data for numerical simulations, scientific visualizations and data analyses. It is most often used to lower I/O and storage costs, and sometimes to lower in‐memory data size as well. With this paper, we consider five categories of data reduction techniques based on their information loss: (1) truly lossless, (2) near lossless, (3) lossy, (4) mesh reduction and (5) derived representations. We then survey available techniques in each of these categories, summarize their properties from a practical point of view and discuss relative merits within a category. We believe, in total, this work will enable simulation scientists and visualization/data analysis scientists to decide which data reduction techniques will be most helpful for their needs.
- 30 March 2018
- Published by Wiley in Computer Graphics Forum
- Vol. 37 (6) , 422-447
Data Reduction Techniques for Simulation, Visualization and Data Analysis
Data reduction is increasingly being applied to scientific data for numerical simulations, scientific visualizations and data analyses. It is most often used to lower I/O and storage costs, and sometimes to lower in‐memory data size as well. With this paper, we consider five categories of data reduction techniques based on their information loss: (1) truly lossless, (2) near lossless, (3) lossy, (4) mesh reduction and (5) derived representations. We then survey available techniques in each of these categories, summarize their properties from a practical point of view and discuss relative merits within a category. We believe, in total, this work will enable simulation scientists and visualization/data analysis scientists to decide which data reduction techniques will be most helpful for their needs.
- 1 January 2018
- Published by Springer Nature
- p. 273-293
Enabling Explorative Visualization with Full Temporal Resolution via In Situ Calculation of Temporal Intervals
- 1 January 2018
- Published by Springer Nature
- p. 273-293
Enabling Explorative Visualization with Full Temporal Resolution via In Situ Calculation of Temporal Intervals