Compression of ERA5 meteorological reanalysis data and their application to simulations with the Lagrangian model for Massive Parallel Trajectory Calculations (MPTRAC v2.7)
Preprint
- 18 July 2025
- preprint
- Published by Copernicus GmbH in EGUsphere
- Vol. 2025, 1-28
Abstract
Computer performance has increased immensely in recent years, but the ability to store data has only increased slightly. The storage requirements for the current version of the ERA5 meteorological reanalysis data provided by the European Centre for Medium-Range Weather Forecasts (ECMWF) have increased by a factor of ∼80 compared to its predecessor ERA-Interim. This presents scientists with major challenges, especially if data covering several decades is to be stored on local computer systems. Accordingly, many compression methods have been developed in recent years with which data can be stored either lossless or lossy. Here we test three of these methods: two lossy compression methods, ZFP and Layer Packing (PCK), and the lossless compressor ZStandard (ZSTD). We investigate how the use of these compressed data affects the results of Lagrangian air parcel trajectory calculations with the Lagrangian model for Massive-Parallel Trajectory Calculations (MPTRAC). We analyzed 10-day forward trajectories that were globally distributed over the free troposphere and stratosphere. The largest transport deviations (up to 1600 km) were derived when using ZFP with the largest compression (CR=25). Using a less strong compression we could reduce the transport deviation (up to 100 km) and still obtain a significant compression (CR=7). Since ZSTD is a lossless compressor, we derive no transport deviations when using these compressed files for trajectory calculations, but do not reduce the use of disk space significantly using this compressor (reduction of ∼30 %, CR=1.5). The best compromise concerning compression efficiency and transport deviations is derived with the layer packing method PCK. The data is compressed by about 50 % (CR=2) but horizontal transport deviations do not exceed 40 km. Thus, our study shows that the PCK compression method would be valuable for application in atmospheric sciences and that with compression of the ERA5 meteorological reanalyses data one can overcome the challenges of high demand of disk space from this data set.Keywords
Funding Information
- Bundesministerium für Forschung, Technologie und Raumfahrt (FKZ: 16ME0670)
This publication has 30 references indexed in Scilit:
- Compressing atmospheric data into its real information contentNature Computational Science, 2021
- The ERA5 global reanalysisQuarterly Journal of the Royal Meteorological Society, 2020
- Error Analysis of ZFP Compression for Floating-Point DataSIAM Journal on Scientific Computing, 2019
- Towards Green Scientific Data Compression Through High-Level I/O Interfaces2018
- Lagrangian transport simulations of volcanic sulfur dioxide emissions: Impact of meteorological data productsJournal of Geophysical Research: Atmospheres, 2016
- Fixed-Rate Compressed Floating-Point ArraysIEEE Transactions on Visualization and Computer Graphics, 2014
- The ERA‐Interim reanalysis: configuration and performance of the data assimilation systemQuarterly Journal of the Royal Meteorological Society, 2011
- Computation, accuracy and applications of trajectories—A review and bibliographyAtmospheric Environment, 1998
- Sensitivity of Three-Dimensional Trajectories to the Spatial and Temporal Densities of the Wind FieldJournal of Applied Meteorology and Climatology, 1990
- The Accuracy of Trajectory Models as Revealed by the Observing System Simulation ExperimentsMonthly Weather Review, 1985