Data-Driven Coordinated Charging for Electric Vehicles With Continuous Charging Rates: A Deep Policy Gradient Approach
- 16 December 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 9 (14) , 12395-12412
Abstract
In this article, we consider a parking lot that manages the charging processes of its parked electric vehicles (EVs). Upon arrival, each EV requests a certain amount of energy. This request should be fulfilled before the EV’s departure. It is of critical importance to coordinate the EVs’ charging rates to smooth out the load profile of the parking lot because inappropriate charging rates can lead to sharp spikes and fluctuations on the load profile, imposing negative effects on the power grid. Meanwhile, empirical studies show that many parking lots exhibit statistical patterns on EV dynamics. For example, the bulk of EVs arrives during rush hours. Therefore, in this article, we incorporate such patterns into charging rate coordination. Although the statistical patterns can be summarized from historical data, they are difficult to be analytically modeled. As a result, we adopt a model-free deep reinforcement learning approach. We also take the latest continuous charging rate control technology into consideration. The decision variables are thus continuous and a policy gradient algorithm is needed to perform reinforcement learning. Technically, we first formulate the problem as a Markov decision process (MDP) with unknown state transition probabilities. To further derive a deep policy gradient algorithm, the challenge lies in the inconsistent and state-dependent action space of the MDP model, due to the constraint to satisfy EVs’ energy demands before their scheduled departure. To tackle the challenge, we design a customized model for neural network training by extending the action space to be consistent and state independent, and revise the reward function to penalize the neural network output if it is beyond the action space of the original MDP model. With this customized model, we then develop a deep policy gradient algorithm based on the proximal policy gradient framework. Numerical results show that our algorithm outperforms the benchmarks.Keywords
Funding Information
- National Sciences and Engineering Research Council of Canada (RGPIN-2017-05853)
- Hong Kong Research Grant Council (RGC) General Research Fund (16202619, 16207318)
- FDCT-MOST Joint Fund Project (0066/2019/AMJ)
- Macao Science and Technology Development Fund (0060/2019/A1, 0162/2019/A3)
This publication has 41 references indexed in Scilit:
- Smoothed Least-laxity-first Algorithm for EV Charging2017
- Robust Online Algorithms for Peak-Minimizing EV Charging Under Multistage UncertaintyIEEE Transactions on Automatic Control, 2017
- Optimal Scheduling for Electric Vehicle Charging With Discrete Charging Levels in Distribution GridIEEE Transactions on Smart Grid, 2016
- Distributed Power Profile Tracking for Heterogeneous Charging of Electric VehiclesIEEE Transactions on Smart Grid, 2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- Scalable Real-Time Electric Vehicles Charging With Discrete Charging RatesIEEE Transactions on Smart Grid, 2015
- Online coordinated charging decision algorithm for electric vehicles without future informationIEEE Transactions on Smart Grid, 2014
- Online welfare maximization for electric vehicle charging with electricity cost2014
- Optimal Scheduling for Charging and Discharging of Electric VehiclesIEEE Transactions on Smart Grid, 2012
- Load Scheduling and Dispatch for Aggregators of Plug-In Electric VehiclesIEEE Transactions on Smart Grid, 2011