Surrogate-free deep reinforcement learning for multiphysics geometric optimization of PEM water electrolyzers

Authors:

Abstract
Proton exchange membrane water electrolyzers (PEMWEs) are inherently constrained by complex multiphysics trade-offs among electrochemical kinetics, two-phase mass transport, and thermal management. The geometric configuration of internal components plays a pivotal role in regulating these coupled processes, yet systematic optimization remains challenging due to the highly nonlinear design space and the substantial computational cost associated with three-dimensional multiphysics simulations. In this study, a surrogate-free optimization framework is developed by directly coupling a Deep Deterministic Policy Gradient (DDPG) reinforcement learning agent with a high-fidelity finite element multiphysics solver. A designed multi-objective reward function is employed to simultaneously enhance current density while constraining pressure drop and peak temperature. This direct interaction enables adaptive exploration of the design space without relying on intermediate approximations. The optimization results demonstrate that the identified configuration improves the average current density from 19837 A/m to 20354 A/m (a 2.61% increase), while promoting a more uniform pressure distribution and maintaining thermal conditions within acceptable operational limits. Furthermore, the results provide insights into the underlying trade-offs among transport processes, highlighting the role of geometric parameters in balancing electrochemical performance and system efficiency. This work demonstrates the feasibility and potential of integrating reinforcement learning with high-fidelity multiphysics simulations for the geometric design of advanced electrochemical energy systems.