Multi-Agent Reinforcement Learning for Adaptive Mesh Refinement
Preprint
Abstract
Adaptive mesh refinement (AMR) is necessary for efficient finite element simulations of complex physical phenomenon, as it allocates limited computational budget based on the need for higher or lower resolution, which varies over space and time. We present a novel formulation of AMR as a fully-cooperative Markov game, in which each element is an independent agent who makes refinement and de-refinement choices based on local information. We design a novel deep multi-agent reinforcement learning (MARL) algorithm called Value Decomposition Graph Network (VDGN), which solves the two core challenges that AMR poses for MARL: posthumous credit assignment due to agent creation and deletion, and unstructured observations due to the diversity of mesh geometries. For the first time, we show that MARL enables anticipatory refinement of regions that will encounter complex features at future times, thereby unlocking entirely new regions of the error-cost objective landscape that are inaccessible by traditional methods based on local error estimators. Comprehensive experiments show that VDGN policies significantly outperform error threshold-based policies in global error and cost metrics. We show that learned policies generalize to test problems with physical features, mesh geometries, and longer simulation times that were not seen in training. We also extend VDGN with multi-objective optimization capabilities to find the Pareto front of the tradeoff between cost and error.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Machine learning mesh-adaptation for laminar and turbulent flows: applications to high-order discontinuous Galerkin solversEngineering with Computers, 2024
- Learning Robust Marking Policies for Adaptive Mesh RefinementSIAM Journal on Scientific Computing, 2024
- Deep reinforcement learning for adaptive mesh refinementJournal of Computational Physics, 2023
- Reinforcement learning for automatic quadrilateral mesh generation: A soft actor–critic approachNeural Networks, 2022
- MFEM: A modular finite element methods libraryComputers & Mathematics with Applications, 2021
- Finite element methods for Maxwell’s equations2020
- A survey and critique of multiagent deep reinforcement learningAutonomous Agents and Multi-Agent Systems, 2019
- Nonconforming Mesh Refinement for High-Order Finite ElementsSIAM Journal on Scientific Computing, 2019
- Adaptive Finite Element Methods for Differential Equations2003