Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning
Authors:
- 30 May 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9403-9409
Abstract
Professional race-car drivers can execute extreme overtaking maneuvers. However, existing algorithms for autonomous overtaking either rely on simplified assumptions about the vehicle dynamics or try to solve expensive trajectory-optimization problems online. When the vehicle approaches its physical limits, existing model-based controllers struggle to handle highly nonlinear dynamics, and cannot leverage the large volume of data generated by simulation or real-world driving. To circumvent these limitations, we propose a new learning-based method to tackle the autonomous overtaking problem. We evaluate our approach in the popular car racing game Gran Turismo Sport, which is known for its detailed modeling of various cars and tracks. By leveraging curriculum learning, our approach leads to faster convergence as well as increased performance compared to vanilla reinforcement learning. As a result, the trained controller outperforms the built-in model-based game AI and achieves comparable over-taking performance with an experienced human driver.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Deriving overtaking strategy from nonlinear model predictive control for a race car2017
- A Survey of Motion Planning and Control Techniques for Self-Driving Urban VehiclesIEEE Transactions on Intelligent Vehicles, 2016
- Reinforcement learning based overtaking decision-making for highway autonomous driving2015
- Towards time-optimal race car driving using nonlinear MPC in real-time2014
- A fast RRT algorithm for motion planning of autonomous road vehicles2014
- Modeling and Nonlinear Adaptive Control for Autonomous Vehicle OvertakingIEEE Transactions on Intelligent Transportation Systems, 2014
- Learning to overtake in TORCS using simple reinforcement learning2010
- Motion planning for urban driving using RRT2008
- Follow the Past: a path-tracking algorithm for autonomous vehiclesInternational Journal of Vehicle Autonomous Systems, 2006
- Efficient Training of Artificial Neural Networks for Autonomous NavigationNeural Computation, 1991