Distributionally Robust Multi-Agent Reinforcement Learning for Intelligent Traffic Control
Preprint
Abstract
Learning-based traffic signal control is typically optimized for average performance under a few nominal demand patterns, which can result in poor behavior under atypical traffic conditions. To address this, we develop a distributionally robust multi-agent reinforcement learning framework for signal control on a 3x3 urban grid calibrated from a contiguous 3x3 subarea of central Athens covered by the pNEUMA trajectory dataset (Barmpounakis and Geroliminis, 2020). Our approach proceeds in three stages. First, we train a baseline multi-agent RL controller in which each intersection is governed by a proximal policy optimization agent with discrete signal phases, using a centralized training, decentralized execution paradigm. Second, to capture demand uncertainty, we construct eight heterogeneous origin-destination-based traffic scenarios-one directly derived from pNEUMA and seven synthetically generated-to span a wide range of spatial and temporal demand patterns. Over this scenario set, we train a contextual-bandit worst-case estimator that assigns mixture weights to estimate adversarial demand distributions conditioned on context. Finally, without modifying the controller architecture, we fine-tune the baseline multi-agent reinforcement learning agents under these estimated worst-case mixtures to obtain a distributionally robust multi-agent reinforcement learning controller. Across all eight scenarios, as well as on an unseen validation network based on the Sioux Falls configuration, the distributionally robust multi-agent reinforcement learning controller consistently reduces horizon-averaged queues and increases average speeds relative to the baseline, achieving up to 51% shorter queues and 38% higher speeds on the worst-performing scenarios.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Impact Of Traffic Congestion on Transportation System: Challenges and Remediations - A reviewMansoura Engineering Journal, 2024
- Learning Decentralized Traffic Signal Controllers With Multi-Agent Graph Reinforcement LearningIEEE Transactions on Mobile Computing, 2023
- Applying the Degree of Urbanisation to the globe: A new harmonised definition reveals a different picture of global urbanisationJournal of Urban Economics, 2021
- Flow: A Modular Learning Framework for Mixed Autonomy TrafficIEEE Transactions on Robotics, 2021
- On the new era of urban traffic monitoring with massive drone data: The pNEUMA large-scale field experimentTransportation Research Part C: Emerging Technologies, 2020
- Adaptive Traffic Control Systems: Domestic and Foreign State of Practice2010
- The Global Burden of Disease Due to Outdoor Air PollutionJournal of Toxicology and Environmental Health, Part A, 2005
- Enriched Sioux Falls scenario with dynamic and disaggregate demand