Abstract
This paper presents a novel model-free deep reinforcement learning (DRL) framework for optimal hydrogen production and delivery operations in a green hydrogen hub (GHH), where produced hydrogen is delivered to customer locations via a fleet of portable tanks. The problem formulation takes into account (1) realistic nonlinear physical constraints and characteristics of the GHH components, such as the proton exchange membrane (PEM) electrolyzer and storage system, and (2) hydrogen delivery challenges, including hydrogen demand and traffic flow uncertainty. The proposed framework adopts 1) a safe proximal policy optimization algorithm that embeds an interior-point log-barrier in the actor loss, which ensures hard operational constraint satisfaction during learning over high-dimensional continuous states and actions, and 2) a deep neural network-based forecasting algorithm that actively monitors traffic flows and travel times between the GHH and consumer locations and estimates hydrogen demand and renewable power generation. Moreover, the DRL method can track electrolyzer degradation by using the predicted remaining useful life of the PEM unit, thereby introducing more realistic conditions that account for the impact of the PEM unit's practical lifetime on GHH operation. The proposed framework effectively schedules the GHH and manages portable hydrogen tank distribution under uncertainties. Comparisons with model-based information-gap decision theory optimization and state-of-the-art DRL methods validate the effectiveness of the proposed framework across various conditions.