OMEGA: Efficient Occlusion-Aware Navigation for Air-Ground Robots in Dynamic Environments via State Space Model
- 16 December 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (2) , 1066-1073
Abstract
Air-ground robots (AGRs) are widely used in surveillance and disaster response due to their exceptional mobility and versatility (i.e., flying and driving). Current AGR navigation systems perform well in static occlusion-prone environments (e.g., indoors) by using 3D semantic occupancy networks to predict occlusions for complete local mapping and then computing Euclidean Signed Distance Field (ESDF) for path planning. However, these systems face challenges in dynamic scenes (e.g., crowds) due to limitations in perception networks' low prediction accuracy and path planners' high computation overhead. In this paper, we propose Omega , which contains O cc M amba with an E fficient A G R-pl A nner to address the above-mentioned problems. OccMamba adopts a novel architecture that separates semantic and occupancy prediction into independent branches, incorporating the mamba module to efficiently extract semantic and geometric features in 3D environments. This ensures the network can learn long-distance dependencies and improve prediction accuracy. Features are then combined within the Bird's Eye View (BEV) space to minimise computational overhead during feature fusion. The resulting semantic occupancy map is integrated into the local map, providing occlusion awareness of the dynamic environment. Our AGR-Planner utilizes this local map and employs Kinodynamic A* search and gradient-based trajectory optimization for ESDF-free and energy-efficient planning. Experiments demonstrate that OccMamba outperforms the state-of-the-art 3D semantic occupancy network with 25.0% mIoU. End-to-end navigation experiments in dynamic scenes verify OMEGA's efficiency, achieving a 96% average planning success rate.Keywords
Funding Information
- National Key R&D Program of China (2022ZD0160201)
- HK RGC RIF (R7030-22)
- HK ITF (GHP/169/20SZ)
- Huawei Flagship Research (2023)
- HK RGC GRF (17208223, 17204424)
- HKU-CAS Joint Laboratory for Intelligent System Software
This publication has 25 references indexed in Scilit:
- OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction2023
- NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space2023
- OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception2023
- MorphoGear: An UAV with Multi-Limb Morphogenetic Gear for Rough-Terrain Locomotion2023
- Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction2023
- VoxFormer: Sparse Voxel Transformer for Camera-Based 3D Semantic Scene Completion2023
- MonoScene: Monocular 3D Semantic Scene Completion2022
- EGO-Swarm: A Fully Autonomous and Decentralized Quadrotor Swarm System in Cluttered Environments2021
- LMSCNet: Lightweight Multiscale 3D Semantic Completion2020
- SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences2019