Strong Scaling for Numerical Weather Prediction at Petascale with the Atmospheric Model NUMA
Preprint
Abstract
Numerical weather prediction (NWP) has proven to be computationally challenging due to its inherent multiscale nature. Currently, the highest resolution NWP models use a horizontal resolution of approximately 15km. At this resolution many important processes in the atmosphere are not resolved. Needless to say this introduces errors. In order to increase the resolution of NWP models highly scalable atmospheric models are needed. The Non-hydrostatic Unified Model of the Atmosphere (NUMA), developed by the authors at the Naval Postgraduate School, was designed to achieve this purpose. NUMA is used by the Naval Research Laboratory, Monterey as the engine inside its next generation weather prediction system NEPTUNE. NUMA solves the fully compressible Navier-Stokes equations by means of high-order Galerkin methods (both spectral element as well as discontinuous Galerkin methods can be used). Mesh generation is done using the p4est library. NUMA is capable of running middle and upper atmosphere simulations since it does not make use of the shallow-atmosphere approximation. This paper presents the performance analysis and optimization of the spectral element version of NUMA. The performance at different optimization stages is analyzed. By using vector intrinsics the main computations reach 1.2 PFlops on the entire machine Mira. The paper also presents scalability studies for two idealized test cases that are relevant for NWP applications. The atmospheric model NUMA delivers an excellent strong scaling efficiency of 99% on the entire supercomputer Mira using a mesh with 1.8 billion grid points. This allows us to run a global forecast of a baroclinic wave test case at 3km uniform horizontal resolution and double precision within the time frame required for operational weather prediction.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Stabilized high-order Galerkin methods based on a parameter-free dynamic SGS model for LESJournal of Computational Physics, 2015
- The quiet revolution of numerical weather predictionNature, 2015
- A Review of Element-Based Galerkin Methods for Numerical Weather Prediction: Finite Elements, Spectral Elements, and Discontinuous GalerkinArchives of Computational Methods in Engineering, 2015
- Early Experience on the Blue Gene/Q Supercomputing System2013
- Comparison between adaptive and uniform discontinuous Galerkin simulations in dry 2D bubble experimentsJournal of Computational Physics, 2013
- Implicit-Explicit Formulations of a Three-Dimensional Nonhydrostatic Unified Model of the Atmosphere (NUMA)SIAM Journal on Scientific Computing, 2013
- Continuous and discontinuous Galerkin methods for a scalable three-dimensional nonhydrostatic atmospheric model: Limited-area modeJournal of Computational Physics, 2012
- p4est: Scalable Algorithms for Parallel Adaptive Mesh Refinement on Forests of OctreesSIAM Journal on Scientific Computing, 2011
- Computational aspects of a scalable high-order discontinuous Galerkin atmospheric dynamical coreComputers & Fluids, 2009
- A study of spectral element and discontinuous Galerkin methods for the Navier–Stokes equations in nonhydrostatic mesoscale atmospheric modeling: Equation sets and test casesJournal of Computational Physics, 2008