Curriculum Reinforcement Learning From Avoiding Collisions to Navigating Among Movable Obstacles in Diverse Environments
- 1 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (5) , 2740-2747
Abstract
Curriculum learning has proven highly effective to speed up training convergence with improved performance in a variety of tasks. Researchers have been studying how a curriculum can be constituted to train reinforcement learning (RL) agents in various application domains. However, discovering curriculum sequencing requires the ranking of sub-tasks or samples in order of difficulty, which is not yet sufficiently studied for robot navigation problems. It is still an open question what navigation strategies can be learned and transferred during multi-stage transfer learning from easy to hard. Furthermore, despite of some attempts of learning real robot manipulation tasks using curriculum, most of existing works are limited to toy or simulated settings rather than realistic scenarios. To address those issues, we first investigated how the model convergence in diverse environments relates to the navigation strategies and difficulty metrics. We found that only some of the environments can be trained from scratch, such as in a relatively open tunnel-like environment that only required wall following. We then carried out a two-stage transfer learning for more difficult environments. We found such approach effective for goal navigation, but failed for more complex tasks where movable obstacles may be on the navigation path. To facilitate more complex policies in the navigation among movable obstacles (NAMO) task, another curriculum with distance and pace functions appropriate to the difficulty of the environment was developed. The proposed scheme was proved effective and the strategies learned were discussed via comprehensive evaluations conducted in simulated and real environments.Keywords
Funding Information
- Taiwan's National Science and Technology Council (111-NU- E-A49-001-NU, 111-2634-F-A49-013, 111-2623-E-A49-007)
- Qualcomm through the Taiwan University Research Collaboration Project
This publication has 21 references indexed in Scilit:
- LeGO-LOAM: Lightweight and Ground-Optimized Lidar Odometry and Mapping on Variable Terrain2018
- Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Mapless Navigation by Leveraging Prior DemonstrationsIEEE Robotics and Automation Letters, 2018
- Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation2017
- From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots2017
- Hierarchical Decision Theoretic Planning for Navigation Among Movable Obstacles2013
- Curriculum learning2009
- Path Planning among Movable Obstacles: A Probabilistically Complete Approach2009
- Planning Among Movable Obstacles with Artificial ConstraintsThe International Journal of Robotics Research, 2008
- Manipulation Planning Among Movable ObstaclesProceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (cat. No.01ch37164), 2007
- Learning and development in neural networks: the importance of starting smallCognition, 1993