Zifan Liu

The volatility of renewable energy generation and nodal electricity prices provides an arbitrage opportunity for Mobile Battery Energy Storage Systems (MBESS) leveraging both temporal and spatial advantages. However, the inherent high complexity and strong randomness of both power and transportation systems lead to complex risks for MBESS control. Existing works mainly consider the market price risk and ignore the transportation system risk caused by traffic congestion. Specifically, they are constrained by two critical limitations: (1) decisions can only be made upon arrival at a destination, making the agent unresponsive on the road, and (2) traffic congestion risk is neither quantified nor controlled, leading to suboptimal routing strategies. To address these limitations, the MBESS needs more flexible “on the road” decision making and multiple risk management capabilities. Guided by this objective, a flexible deep reinforcement learning-based MBESS control framework is proposed, considering both market and traffic congestion risk. First, dynamic routing ability is integrated with the MBESS agent to provide more flexibility in making decisions, regardless of whether the agent has reached the designated location or not. Second, two risk metrics are proposed to quantitatively assess the traffic congestion risk based on moving time, and then the agent can make decisions considering both market and traffic congestion risk. Finally, considering the inefficiency of learning caused by introducing multiple risks, a risk curriculum learning method is proposed to improve the training efficiency and reduce learning costs. These components are unified in the Multiple Risk Estimation SDDPG (MRE-SDDPG) algorithm, which jointly maximizes profitability while controlling electricity price and traffic congestion risk. Simulations in the IEEE 30 bus environment show that the proposed framework can increase profit by 8.6% while reducing the traffic time by 15.8% on average, demonstrating the superiority of our design in considering traffic congestion risk.

The mobile battery energy storage systems (MBESS) utilize flexibility in temporal and spatial to enhance smart grid resilience and economic benefits. Recently, the high penetration of renewable energy increases the volatility of electricity prices and gives MBESS an opportunity for price difference arbitrage. However, the strong randomness of both the traffic system and renewable energy leads to difficulty in achieving profit with acceptable risk. To address this problem, this paper proposes a risk-sensitive MBESS control framework based on safe deep reinforcement learning, which can constrain the risk under a certain level according to the company’s risk preference. The risk-estimation safe deep deterministic policy gradient (RE-SDDPG) algorithm is proposed to learn the optimal policy under the premise of lacking direct risk signals. Moreover, a hybrid risk estimation method is proposed to avoid local convergence caused by inaccurate estimation during the learning process. Last, a parameter-sharing method is applied to increase learning efficiency by sharing the Q networks’ parameters. The proposed methods are tested in the IEEE 30-bus system. The results show that the proposed method can keep the profit at a relatively high level while reducing the risk and increase learning efficiency compared with existing methods.