Robust Dynamic Bus Control: a Distributional Multi-Agent Reinforcement Learning Approach
- 20 December 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 24 (4) , 4075-4088
Abstract
The bus system is a critical component of sustainable urban transportation. However, the operation of a bus fleet is unstable in nature, and bus bunching has become a common phenomenon that undermines the efficiency and reliability of bus systems. Recently research has demonstrated the promising application of multi-agent reinforcement learning (MARL) to achieve efficient vehicle holding control to avoid bus bunching. However, existing studies essentially overlook the robustness issue resulting from perturbations and anomalies in a transit system, which is of utmost importance when transferring the models for real-world deployment/application. In this study, we integrate implicit quantile network and meta-learning to develop a distributional MARL framework—IQNC-M—to learn continuous control. The proposed IQNC-M framework achieves efficient and reliable control decisions through better handling various uncertainties in real-time transit operations. Specifically, we introduce an interpretable meta-learning module to incorporate global information into the distributional MARL framework, which is an effective solution to circumvent the credit assignment issue in the transit system. In addition, we design a specific learning procedure to train each agent within the framework to pursue a robust control policy. We develop simulation environments based on real-world bus services and passenger demand data and evaluate the proposed framework against both traditional holding control models and state-of-the-art MARL models. Our results show that the proposed IQNC-M framework can effectively handle the general perturbations and various extreme events, such as traffic state perturbations and demand surges, thus improving both efficiency and reliability of the transit system.Keywords
All Related Versions
Funding Information
- Fonds de Recherche du Québec-Société et Culture (FRQSC) under the NSFC-FRQSC Research Program on Smart Cities and Big Data
- Canada Foundation for Innovation (CFI) John R. Evans Leaders Fund
This publication has 30 references indexed in Scilit:
- Remarks on quantiles and distortion risk measuresEuropean Actuarial Journal, 2012
- Dynamic bus holding strategies for schedule reliability: Optimal linear control and performance analysisTransportation Research Part B: Methodological, 2011
- Impacts of Holding Control Strategies on Transit PerformanceTransportation Research Record: Journal of the Transportation Research Board, 2011
- A headway-based approach to eliminate bus bunching: Systematic analysis and comparisonsTransportation Research Part B: Methodological, 2009
- Real-Time Control of Buses in a Transit Corridor Based on Vehicle Holding and Boarding LimitsTransportation Research Record: Journal of the Transportation Research Board, 2009
- An Analytic Stochastic Model for the Transit Vehicle Holding ProblemTransportation Science, 2001
- The Holding Problem with Real–Time Information AvailableTransportation Science, 2001
- A Class of Distortion Operators for Pricing Financial and Insurance RisksJournal of Risk and Insurance, 2000
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Robust Estimation of a Location ParameterThe Annals of Mathematical Statistics, 1964