Model-Free UAV Navigation in Unknown Complex Environments Using Vision-Based Reinforcement Learning
Open Access
Abstract
Autonomous UAV navigation in unknown and complex environments remains a core challenge, especially under limited sensing and computing resources. While most methods rely on modular pipelines involving mapping, planning, and control, they often suffer from poor real-time performance, limited adaptability, and high dependency on accurate environment models. Moreover, many deep-learning-based solutions either use RGB images prone to visual noise or optimize only a single objective. In contrast, this paper proposes a unified, model-free vision-based DRL framework that directly maps onboard depth images and UAV state information to continuous navigation commands through a single convolutional policy network. This end-to-end architecture eliminates the need for explicit mapping and modular coordination, significantly improving responsiveness and robustness. A novel multi-objective reward function is designed to jointly optimize path efficiency, safety, and energy consumption, enabling adaptive flight behavior in unknown complex environments. The trained policy demonstrates generalization in diverse simulated scenarios and transfers effectively to real-world UAV flights. Experiments show that our approach achieves stable navigation and low latency.Keywords
Funding Information
- New Energy and Industrial Technology Development Organization (NEDO) (JPNP22002)
This publication has 60 references indexed in Scilit:
- UAV trajectory tracking under wind disturbance based on novel antidisturbance sliding mode controlAerospace Science and Technology, 2024
- Q-learning-based unmanned aerial vehicle path planning with dynamic obstacle avoidanceApplied Soft Computing, 2023
- PID control of quadrotor UAVs: A surveyAnnual Reviews in Control, 2023
- Techniques, Answers, and Real-World UAV Implementations for Precision FarmingWireless Personal Communications, 2023
- Mapless navigation for UAVs via reinforcement learning from demonstrationsScience China Technological Sciences, 2023
- Unmanned aerial vehicles (UAVs): practical aspects, applications, open challenges, security issues, and future trendsIntelligent Service Robotics, 2023
- A hybrid optimization framework for UAV reconnaissance mission planningComputers & Industrial Engineering, 2022
- Autonomous Obstacle Avoidance and Target Tracking of UAV Based on Deep Reinforcement LearningJournal of Intelligent & Robotic Systems, 2022
- Ensemble selector for attribute reductionApplied Soft Computing, 2018
- A Weighted Measurement Fusion Kalman Filter implementation for UAV navigationAerospace Science and Technology, 2012