Efficient Frontier Selection via Reinforcement Learning for Exploring Unstructured Environments with Minimal Sensing
Open Access
- 16 June 2026
- journal article
- research article
- Published by MDPI AG in Technologies
- Vol. 14 (6) , 365
Abstract
In recent years, reinforcement learning (RL) has been applied to frontier-based exploration to enhance a robot’s decision-making policy and improve exploration performance. In this work, we address this scenario with the aim of pushing forward the finding of the optimal frontier selection policy in unknown, unstructured environments, with RL deployed for a minimal sensing drone setup. We propose a novel policy architecture, featuring an attention module that uses the global map features captured by a convolutional neural network together with local frontier features in the form of scalar values, trained end-to-end with a scoring network using the Proximal Policy Optimization algorithm over a 2D randomized unstructured environment. Our approach demonstrates improved exploration efficiency in the evaluated scenarios, as it surpasses purely heuristic-based frontier selection strategies used as baselines for other RL methods, achieving shorter paths than the Nearest Frontier, the Hybrid Approach, and the TARE local horizon, as well as one-shot sim-to-real policy deployment.Keywords
All Related Versions
Funding Information
- Comunidad de Madrid (PIPF-2024/TEC-34898 (BOCM-5021/2025))
This publication has 26 references indexed in Scilit:
- Exploring Unstructured Environments Using Minimal Sensing on Cooperative Nano-DronesIEEE Robotics and Automation Letters, 2024
- An improved frontier-based robot exploration strategy combined with deep reinforcement learningRobotics and Autonomous Systems, 2024
- Deep Reinforcement Learning-Based Large-Scale Robot ExplorationIEEE Robotics and Automation Letters, 2024
- Hierarchical path planner for unknown space exploration using reinforcement learning-based intelligent frontier selectionExpert Systems with Applications, 2023
- Reinforcement learning in robotic applications: a comprehensive surveyArtificial Intelligence Review, 2021
- Frontier Detection and Reachability Analysis for Efficient 2D Graph-SLAM Based Active Exploration2020
- Robotic Exploration of Unknown 2D Environment Using a Frontier-based Automatic-Differentiable Information Gain Measure2020
- Deep Reinforcement Learning Robot for Search and Rescue Applications: Exploration in Unknown Cluttered EnvironmentsIEEE Robotics and Automation Letters, 2019
- Incremental algorithms for Safe and Reachable Frontier Detection for robot explorationRobotics and Autonomous Systems, 2015
- Exploration strategies based on multi-criteria decision making for searching environments in rescue operationsAutonomous Robots, 2011