Struct-MDC: Mesh-Refined Unsupervised Depth Completion Leveraging Structural Regularities From Visual SLAM
- 3 May 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (3) , 6391-6398
Abstract
Feature-based visual simultaneous localization and mapping (SLAM) methods only estimate the depth of extracted features, generating a sparse depth map. To solve this sparsity problem, depth completion tasks that estimate a dense depth from a sparse depth have gained significant importance in robotic applications like exploration. Existing methodologies that use sparse depth from visual SLAM mainly employ point features. However, point features have limitations in preserving structural regularities owing to texture-less environments and sparsity problems. To deal with these issues, we perform depth completion with visual SLAM using line features, which can better contain structural regularities than point features. The proposed methodology creates a convex hull region by performing constrained Delaunay triangulation with depth interpolation using line features. However, the generated depth includes low-frequency information and is discontinuous at the convex hull boundary. Therefore, we propose a mesh depth refinement (MDR) module to address this problem. The MDR module effectively transfers the high-frequency details of an input image to the interpolated depth and plays a vital role in bridging the conventional and deep learning-based approaches. The Struct-MDC outperforms other state-of-the-art algorithms on public and our custom datasets, and even outperforms supervised methodologies for some metrics. In addition, the effectiveness of the proposed MDR module is verified by a rigorous ablation study.Keywords
All Related Versions
Funding Information
- Unmanned Swarm CPS Research Laboratory Program
- Defense Acquisition Program Administration
- Agency for Defense Development (911257202)
This publication has 35 references indexed in Scilit:
- VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State EstimatorIEEE Transactions on Robotics, 2018
- Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image2018
- Unsupervised Learning of Depth and Ego-Motion from Video2017
- PL-SVO: Semi-direct Monocular Visual Odometry by combining points and line segments2016
- Measuring the performance of single image depth estimation methods2016
- Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation2013
- Indoor Segmentation and Support Inference from RGBD Images2012
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Constrained delaunay triangulationsAlgorithmica, 1989
- Random sample consensusCommunications of the ACM, 1981