Model inductive bias enhanced deep reinforcement learning for robot navigation in crowded environments
Open Access
- 2 July 2024
- journal article
- research article
- Published by Springer Nature in Complex & Intelligent Systems
- Vol. 10 (5) , 6965-6982
Abstract
Navigating mobile robots in crowded environments poses a significant challenge and is essential for the coexistence of robots and humans in future intelligent societies. As a pragmatic data-driven approach, deep reinforcement learning (DRL) holds promise for addressing this challenge. However, current DRL-based navigation methods have possible improvements in understanding agent interactions, feedback mechanism design, and decision foresight in dynamic environments. This paper introduces the model inductive bias enhanced deep reinforcement learning (MIBE-DRL) method, drawing inspiration from a fusion of data-driven and model-driven techniques. MIBE-DRL extensively incorporates model inductive bias into the deep reinforcement learning framework, enhancing the efficiency and safety of robot navigation. The proposed approach entails a multi-interaction network featuring three modules designed to comprehensively understand potential agent interactions in dynamic environments. The pedestrian interaction module can model interactions among humans, while the temporal and spatial interaction modules consider agent interactions in both temporal and spatial dimensions. Additionally, the paper constructs a reward system that fully accounts for the robot’s direction and position factors. This system's directional and positional reward functions are built based on artificial potential fields (APF) and navigation rules, respectively, which can provide reasoned evaluations for the robot's motion direction and position during training, enabling it to receive comprehensive feedback. Furthermore, the incorporation of Monte-Carlo tree search (MCTS) facilitates the development of a foresighted action strategy, enabling robots to execute actions with long-term planning considerations. Experimental results demonstrate that integrating model inductive bias significantly enhances the navigation performance of MIBE-DRL. Compared to state-of-the-art methods, MIBE-DRL achieves the highest success rate in crowded environments and demonstrates advantages in navigation time and maintaining a safe social distance from humans.Keywords
This publication has 37 references indexed in Scilit:
- UAV 3D online track planning based on improved SAC algorithmJournal of the Brazilian Society of Mechanical Sciences and Engineering, 2023
- Generalized multi-agent competitive reinforcement learning with differential augmentationExpert Systems with Applications, 2023
- An Efficiently Convergent Deep Reinforcement Learning-Based Trajectory Planning Method for Manipulators in Dynamic EnvironmentsJournal of Intelligent & Robotic Systems, 2023
- Balanced incremental deep reinforcement learning based on variational autoencoder data augmentation for customer credit scoringEngineering Applications of Artificial Intelligence, 2023
- Multi-level context-driven interaction modeling for human future trajectory predictionNeural Computing and Applications, 2022
- Robot navigation in a crowd by integrating deep reinforcement learning and online planningApplied Intelligence, 2022
- Deep reinforcement learning in medical imaging: A literature reviewMedical Image Analysis, 2021
- Incorporating Actor-Critic in Monte Carlo tree search for symbolic regressionNeural Computing and Applications, 2021
- Mobile robot path planning using membrane evolutionary artificial potential fieldApplied Soft Computing, 2019
- Reciprocal n-Body Collision Avoidance2011