A Hierarchical Deep Reinforcement Learning Framework With High Efficiency and Generalization for Fast and Safe Navigation
Open Access
- 20 July 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Electronics
- Vol. 70 (5) , 4962-4971
Abstract
We present a hierarchical deep reinforcement learning (DRL) framework with prominent sampling efficiency and sim-to-real transfer ability for fast and safe navigation: the low-level DRL policy enables the robot to move toward the target position and keep a safe distance to obstacles simultaneously; the high-level DRL policy is supplemented to further enhance the navigation safety. We select a waypoint located on the path from the robot to the ultimate goal as the subgoal to reduce the state space and avoid sparse reward. Moreover, the path is generated based on either a local or a global map, which can significantly improve the sampling efficiency, safety, and generalization ability of the proposed DRL framework. Additionally, a target-directed representation for the action space can be derived based on the subgoal to improve the motion efficiency and reduce the action space. In order to demonstrate the eminent sampling efficiency, motion performance, obstacle avoidance, and generalization ability of the proposed framework, we implement sufficient comparisons with the nonlearning navigation methods and DRL-based baselines, with videos, data, code, and other supplemental material shown on our website.Keywords
Funding Information
- Japan Society for the Promotion of Science (JP20KK0256)
This publication has 29 references indexed in Scilit:
- Target-driven visual navigation in indoor scenes using deep reinforcement learning2017
- Human-level control through deep reinforcement learningNature, 2015
- Reciprocal collision avoidance with acceleration-velocity obstacles2011
- Reciprocal n-Body Collision Avoidance2011
- Simultaneous localization and mapping (SLAM): part IIIEEE Robotics & Automation Magazine, 2006
- Map-based navigation in mobile robots:Cognitive Systems Research, 2003
- Vision for mobile robot navigation: a surveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- RRT-connect: An efficient approach to single-query path planning2000
- Real-time map building and navigation for autonomous robots in unknown environmentsIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1998
- The dynamic window approach to collision avoidanceIEEE Robotics & Automation Magazine, 1997