Reinforcement Learning Aided Sequential Optimization for Unsignalized Intersection Management of Robot Traffic
Open Access
- 29 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 104052-104070
Abstract
We consider the problem of optimal unsignalized intersection management, wherein we seek to obtain safe and optimal trajectories, for a set of robots that arrive randomly and continually. This problem involves repeatedly solving a mixed integer program (with robot acceleration trajectories as decision variables) with different parameters, for which the computation time using a naive optimization algorithm scales exponentially with the number of robots and lanes. Hence, such an approach is not suitable for real-time implementation. In this paper, we propose a solution framework that combines learning and sequential optimization. In particular, we propose an algorithm for learning a shared policy that given the traffic state information, determines the crossing order of the robots. Then, we optimize the trajectories of the robots sequentially according to that crossing order. This approach inherently guarantees safety at all times. We validate the performance of this approach using extensive simulations and compare our approach against 5 different heuristics from the literature in 9 different simulation settings . Our approach, on average, significantly outperforms the heuristics from the literature in various metrics like objective function, weighted average of crossing times and computation time. For example, in some scenarios, we have observed that our approach offers up to 150% improvement in objective value over the first come first serve heuristic. Even on untrained scenarios, our approach shows a consistent improvement (in objective value) of more than 30% over all heuristics under consideration. We also show through simulations that the computation time for our approach scales linearly with the number of robots (assuming all other factors are constant) . We further implement the learnt policies on physical robots with a few modifications to the solution framework to address real-world challenges and establish its real-time implementability.Keywords
All Related Versions
Funding Information
- Nokia Corporate Social Responsibility (CSR) grant through the Nokia Centre of Excellence in Networked Robotics at Indian Institute of Science
This publication has 58 references indexed in Scilit:
- Improved reservation-based method with controllable gap strategy for vehicle coordination at non-signalized intersectionsPhysica A: Statistical Mechanics and its Applications, 2022
- Real-Time Trajectory Planning for Autonomous Driving with Gaussian Process and Incremental Refinement2022
- Learning From Oracle Demonstrations—A New Approach to Develop Autonomous Intersection Management Control Algorithms Based on Multiagent Deep Reinforcement LearningIEEE Access, 2022
- A Comprehensive Survey on Cooperative Intersection Management for Heterogeneous Connected VehiclesIEEE Access, 2022
- Reinforcement learning for combinatorial optimization: A surveyComputers & Operations Research, 2021
- Distributed Multi-Agent Navigation Based on Reciprocal Collision Avoidance and Locally Confined Multi-Agent Path Finding2021
- Optimal time trajectory and coordination for connected and automated vehiclesAutomatica, 2021
- DCL-AIM: Decentralized coordination learning of autonomous intersection management for connected and automated vehiclesTransportation Research Part C: Emerging Technologies, 2019
- A decentralized energy-optimal control framework for connected automated vehicles at signal-free intersectionsAutomatica, 2018
- Sustainability assessments of cooperative vehicle intersection control at an urban corridorTransportation Research Part C: Emerging Technologies, 2013