AAV Visual Navigation in the Large-Scale Outdoor Environment: A Semantic-Map-Based Cognitive Escape Reinforcement Learning Method
- 20 January 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (11) , 15926-15938
Abstract
The rapid development of the unmanned aerial vehicle (UAV) technology has significantly facilitated its application in the burgeoning Internet of Things (IoT) ecosystem. UAV visual navigation has emerged as a particularly vibrant and crucial research area, holding the potential to greatly enhance automated IoT services. Deep reinforcement learning (DRL), recognized as an effective method for visual navigation, encounters two significant challenges in the complex outdoor environments, i.e., the partial observability and the local optima trapping. In this paper, to address these two challenges, we propose a novel semantic map-based cognitive escape reinforcement learning (SM-CERL) navigation method, which consists of two innovatively designed modules, namely, the semantic mapping module (SMM) and the cognitive escape module (CEM). By deeply exploring the similarity structure of raw images and mapping them to the advanced semantic representations, the SMM constructs a semantic map with rich implications, which provides the UAV with the memory to enhance its understanding of the environment. Meanwhile, the CEM can proactively identify local optima and leverage the rule knowledge to establish efficient escape strategies. The holistic fusion of semantic mapping and cognitive escape mechanisms efficiently enhances the environmental comprehension and prevents UAVs from being trapped in local optima. Extensive experimental results demonstrate that our SM-CERL outperforms the existing classical and state-of-the-art methods in terms of the navigation accuracy and efficiency.Keywords
Funding Information
- Jiangsu Province Frontier Leading Technology Basic Research Project (BK20222013)
- National Natural Science Foundation of China (62222107, 62231015)
This publication has 31 references indexed in Scilit:
- Densifying SLAM for UAV Navigation by Fusion of Monocular Depth Prediction2023
- Improving Target-driven Visual Navigation with Attention on 3D Spatial RelationshipsNeural Processing Letters, 2022
- Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation2021
- First return, then exploreNature, 2021
- Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks2019
- Vector-based navigation using grid-like representations in artificial agentsNature, 2018
- A survey on vision-based UAV navigationGeo-spatial Information Science, 2018
- Curiosity-Driven Exploration by Self-Supervised Prediction2017
- Hybrid computing using a neural network with dynamic external memoryNature, 2016
- Human-level control through deep reinforcement learningNature, 2015