Smart Train Operation Algorithms Based on Expert Knowledge and Reinforcement Learning
- 20 July 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 52 (2) , 716-727
Abstract
During decades, the automatic train operation (ATO) system has been gradually adopted in many subway systems for its low-cost and intelligence. This article proposes two smart train operation (STO) algorithms by integrating the expert knowledge with reinforcement learning algorithms. Compared with previous works, the proposed algorithms can realize the control of continuous action for the subway system and optimize multiple critical objectives without using an offline speed profile. First, through learning historical data of experienced subway drivers, we extract the expert knowledge rules and build inference methods to guarantee the riding comfort, the punctuality, and the safety of the subway system. Then we develop two algorithms for optimizing the energy efficiency of train operation. One is the STO algorithm based on deep deterministic policy gradient named (STOD) and the other is the STO algorithm based on normalized advantage function (STON). Finally, we verify the performance of proposed algorithms via some numerical simulations with the real field data from the Yizhuang Line of the Beijing Subway and illustrate that the developed STO algorithm are better than expert manual driving and existing ATO algorithms in terms of energy efficiency. Moreover, STOD and STON can adapt to different trip times and different resistance conditions.Keywords
All Related Versions
Funding Information
- Key project of the National Natural Science Foundation of China (61936009)
- National Major Research Program (2018AAA0101604)
- Department of Automation, Tsinghua University, under the supervision of Prof. Shiji Song and Dr. Keyou You
This publication has 34 references indexed in Scilit:
- Energy-efficient train control: From local convexity to global optimization and uniquenessAutomatica, 2013
- Optimizing trains movement on a railway networkOmega, 2012
- Optimization of Train Speed Profile for Minimum Energy ConsumptionIEEJ Transactions on Electrical and Electronic Engineering, 2010
- Application of reinforcement learning in robot soccerEngineering Applications of Artificial Intelligence, 2007
- A factorial analysis for the determination of an optimal train speed with a desired ride comfortApplied Acoustics, 2005
- Power Systems Stability Control: Reinforcement Learning FrameworkIEEE Transactions on Power Systems, 2004
- A review of methods to measure and calculate train resistancesProceedings of the Institution of Mechanical Engineers, Part F: Journal of Rail and Rapid Transit, 2000
- On an optimal control problem of train operationIEEE Transactions on Automatic Control, 2000
- A survey of tools for the validation and verification of knowledge-based systems: 1985–1995Decision Support Systems, 1997
- A Survey of Longitudinal Acceleration Comfort Studies in Ground Transportation VehiclesJournal of Dynamic Systems, Measurement, and Control, 1977