Performance-portable Numerical Relativity with AthenaK
Open Access
- 1 June 2025
- journal article
- Published by American Astronomical Society in The Astrophysical Journal Supplement Series
- Vol. 278 (2) , 50
Abstract
We present the numerical relativity module within AthenaK, an open-source performance-portable astrophysics code designed for exascale computing applications. This module employs the Z4c formulation to solve the Einstein equations. We demonstrate its accuracy through a series of standard numerical relativity tests, including convergence of the gravitational waveform from binary black hole coalescence. Furthermore, we conduct scaling tests on OLCF Frontier, NERSC Perlmutter, and ALCF Aurora, where AthenaK exhibits excellent weak-scaling efficiency of 80% on up to 65,536 AMD MI250X GPUs on Frontier (relative to four GPUs) and 67% on Aurora up to 24,576 Intel Data Center Max Series GPUs (relative to 12 GPUs) and strong-scaling efficiencies of 84% and 77% on AMD MI250X and NVIDIA A100 GPUs on Frontier and Perlmutter, respectively. Additionally, we observe a significant performance boost, with 2 orders of magnitude speedup (≳200×) on a GPU compared to a single CPU core, affirming that AthenaK is well suited for exascale computing, and thereby expanding the potential for breakthroughs in numerical relativity research.Keywords
Funding Information
- U.S. Department of Energy (DE-SC0021177)
- U.S. Department of Energy (E-SC0024388)
- National Aeronautics and Space Administration (80NSSC21K172)
- National Science Foundation (PHY-2011725)
- National Science Foundation (PHY- 2020275)
- National Science Foundation (PHY-2116686)
- National Science Foundation (AST-2108467)
- National Science Foundation (2139536)
- EU H2020 Program (inGraSp-714626)
- EU Horizon program (InspiReM-101043372)
- U.S. Department of Energy (DE-AC02- 05CH11231)
- U.S. Department of Energy (DE-AC05-00OR22725)
This publication has 76 references indexed in Scilit:
- Computational challenges in numerical relativity in the gravitational-wave eraNature Computational Science, 2021
- A fourth-order accurate finite volume method for ideal MHD via upwind constrained transportJournal of Computational Physics, 2018
- Prospects for observing and localizing gravitational-wave transients with Advanced LIGO, Advanced Virgo and KAGRALiving Reviews in Relativity, 2018
- SpECTRE: A task-based discontinuous Galerkin code for relativistic astrophysicsJournal of Computational Physics, 2017
- Extraction of gravitational waves in numerical relativityLiving Reviews in Relativity, 2016
- Numerical Relativity2010
- Numerical relativity: a reviewClassical and Quantum Gravity, 2001
- Local adaptive mesh refinement for shock hydrodynamicsJournal of Computational Physics, 1989
- On the hyperbolicity of Einstein's and other gauge field equationsCommunications in Mathematical Physics, 1985
- Adaptive mesh refinement for hyperbolic partial differential equationsJournal of Computational Physics, 1984