Abstract
In recent decades, modern power systems have seen a significant increase in the integration of renewable energy resources. These changes bring numerous benefits, including mitigating climate change and accelerating the transition to a low-carbon economy. However, the inherent instability of renewable energy sources presents new challenges for both the planning and operation of power systems. Traditionally, many scientific studies have focused on developing model-based optimization methods for electric vehicle (EV) dispatch problems. However, these methods often fall short in effectively handling the highly dynamic and stochastic nature of power systems due to their complexity. Reinforcement learning (RL), a model-free approach, offers a solution by capturing various uncertainties through extensive interactions with the environment and adapting to diverse state conditions in real-time. As such, reinforcement learning algorithms are well-suited for addressing EV dispatch problems. EVs can participate in vehicle-to-grid (V2G) operations, where they charge from the grid and discharge power back into it, providing valuable ancillary services such as overload reduction and energy imbalance management. Optimizing the scheduling of EV charging and discharging is crucial for maximizing the benefits to EV owners. Given the fluctuating electricity prices driven by the unstable nature of renewable energy sources, various RL algorithms, including Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed DDPG (TD3), will be applied to V2G problems to minimize electricity costs for EV owners.

This publication has 8 references indexed in Scilit: