Performance-portable Binary Neutron Star Mergers with AthenaK
Open Access
- 14 January 2025
- journal article
- research article
- Published by American Astronomical Society in The Astrophysical Journal Supplement Series
- Vol. 276 (2) , 35
Abstract
We introduce an extension to the AthenaK code for general-relativistic magnetohydrodynamics (GRMHD) in dynamical spacetimes using a 3+1 conservative Eulerian formulation. Like the fixed-spacetime GRMHD solver, we use standard finite-volume methods to evolve the fluid and a constrained-transport scheme to preserve the divergence-free constraint for the magnetic field. We also utilize a first-order flux correction (FOFC) scheme to reduce the need for an artificial atmosphere and optionally enforce a maximum principle to improve robustness. We demonstrate the accuracy of AthenaK using a set of standard tests in flat and curved spacetimes. Using a SANE accretion disk around a Kerr black hole, we compare the new solver to the existing solver for stationary spacetimes using the so-called “HARM-like” formulation. We find that both formulations converge to similar results. We also include the first published binary neutron star (BNS) mergers performed on graphical processing units (GPUs). Thanks to the FOFC scheme, our BNS mergers maintain a relative error of O ( 1 0 − 11 ) or better in baryon mass conservation up to collapse. Finally, we perform scaling tests of AthenaK on OLCF Frontier, where we show excellent weak scaling of ≥80% efficiency up to 32,768 GPUs and 74% up to 65,536 GPUs for a GRMHD problem in dynamical spacetimes with six levels of mesh refinement. AthenaK achieves an order-of-magnitude speedup using GPUs compared to CPUs, demonstrating that it is suitable for performing numerical relativity problems on modern exascale resources.Keywords
All Related Versions
Funding Information
- DOE ∣ SC ∣ Nuclear Physics (DE-SC0021177)
- DOE ∣ SC ∣ Nuclear Physics (DE-SC0024388)
- National Aeronautics and Space Administration (80NSSC21K1720)
- National Science Foundation (PHY-2011725)
- National Science Foundation (PHY-2020275)
- National Science Foundation (PHY-2116686)
- National Science Foundation (AST-2108467)
- EC ∣ ERC ∣ HORIZON EUROPE European Research Council (InspiReM-101043372)
This publication has 97 references indexed in Scilit:
- High Order Strong Stability Preserving Time DiscretizationsJournal of Scientific Computing, 2008
- A limiter for PPM that preserves accuracy at smooth extremaJournal of Computational Physics, 2008
- An unsplit Godunov method for ideal MHD via constrained transport in three dimensionsJournal of Computational Physics, 2008
- An improved weighted essentially non-oscillatory scheme for hyperbolic conservation lawsJournal of Computational Physics, 2008
- An unsplit Godunov method for ideal MHD via constrained transportJournal of Computational Physics, 2005
- On Godunov-type methods near low densitiesJournal of Computational Physics, 1991
- Flux-corrected transport techniques for multidimensional compressible magnetohydrodynamicsJournal of Computational Physics, 1991
- The Piecewise Parabolic Method (PPM) for gas-dynamical simulationsJournal of Computational Physics, 1984
- Adaptive mesh refinement for hyperbolic partial differential equationsJournal of Computational Physics, 1984
- Towards the ultimate conservative difference scheme. II. Monotonicity and conservation combined in a second-order schemeJournal of Computational Physics, 1974