Learning to Play Soccer by Reinforcement and Applying Sim-to-Real to Compete in the Real World
Authors:
- Hansenclever F. BassaniHBHansenclever F. Bassani
- Renie A. DelgadoRDRenie A. Delgado
- Jose Nilton de O. Lima JuniorJJJose Nilton de O. Lima Junior
- Heitor R. MedeirosHMHeitor R. Medeiros
- Pedro H. M. BragaPBPedro H. M. Braga
- Alain TappATAlain Tapp
Preprint
Abstract
This work presents an application of Reinforcement Learning (RL) for the complete control of real soccer robots of the IEEE Very Small Size Soccer (VSSS), a traditional league in the Latin American Robotics Competition (LARC). In the VSSS league, two teams of three small robots play against each other. We propose a simulated environment in which continuous or discrete control policies can be trained, and a Sim-to-Real method to allow using the obtained policies to control a robot in the real world. The results show that the learned policies display a broad repertoire of behaviors that are difficult to specify by hand. This approach, called VSSS-RL, was able to beat the human-designed policy for the striker of the team ranked 3rd place in the 2018 LARC, in 1-vs-1 matches.Keywords
All Related Versions
This publication has 3 references indexed in Scilit:
- Challenges of real-world reinforcement learning: definitions, benchmarks and analysisMachine Learning, 2021
- Google Research Football: A Novel Reinforcement Learning EnvironmentProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Learning dexterous in-hand manipulationThe International Journal of Robotics Research, 2019