SPDepth: Enhancing Self-Supervised Indoor Monocular Depth Estimation via Self-Propagation
Open Access
- 16 October 2024
- journal article
- research article
- Published by MDPI AG in Future Internet
- Vol. 16 (10) , 375
Abstract
Due to the existence of low-textured areas in indoor scenes, some self-supervised depth estimation methods have specifically designed sparse photometric consistency losses and geometry-based losses. However, some of the loss terms cannot supervise all the pixels, which limits the performance of these methods. Some approaches introduce an additional optical flow network to provide dense correspondences supervision, but overload the loss function. In this paper, we propose to perform depth self-propagation based on feature self-similarities, where high-accuracy depths are propagated from supervised pixels to unsupervised ones. The enhanced self-supervised indoor monocular depth estimation network is called SPDepth. Since depth self-similarities are significant in a local range, a local window self-attention module is embedded at the end of the network to propagate depths in a window. The depth of a pixel is weighted using the feature correlation scores with other pixels in the same window. The effectiveness of self-propagation mechanism is demonstrated in the experiments on the NYU Depth V2 dataset. The root-mean-squared error of SPDepth is 0.585 and the δ1 accuracy is 77.6%. Zero-shot generalization studies are also conducted on the 7-Scenes dataset and provide a more comprehensive analysis about the application characteristics of SPDepth.Keywords
Funding Information
- Application Innovation Project of CASC (KZ36006202, 2023YFC3300029, LD24F020007, Jxsg2023102268)
- National Key Research and Development Program of China (KZ36006202, 2023YFC3300029, LD24F020007, Jxsg2023102268)
- Zhejiang Provincial Natural Science Foundation of China (KZ36006202, 2023YFC3300029, LD24F020007, Jxsg2023102268)
- “One Thousand Plan” projects in Jiangxi Province (KZ36006202, 2023YFC3300029, LD24F020007, Jxsg2023102268)
This publication has 47 references indexed in Scilit:
- BinsFormer: Revisiting Adaptive Bins for Monocular Depth EstimationIEEE Transactions on Image Processing, 2024
- FDepth: Self-supervised indoor monocular depth estimation via optical flow consistency and feature map synthesisEngineering Applications of Artificial Intelligence, 2024
- DepthFormer: Exploiting Long-range Correlation and Local Information for Accurate Monocular Depth EstimationMachine Intelligence Research, 2023
- Unsupervised Monocular Estimation of Depth and Visual Odometry Using Attention and Depth-Pose Consistency LossIEEE Transactions on Multimedia, 2023
- URCDC-Depth: Uncertainty Rectified Cross-Distillation With CutFlip for Monocular Depth EstimationIEEE Transactions on Multimedia, 2023
- FsaNet: Frequency Self-Attention for Semantic SegmentationIEEE Transactions on Image Processing, 2023
- Self-Supervised Monocular Depth Estimation With Multiscale PerceptionIEEE Transactions on Image Processing, 2022
- ADAADepth: Adapting Data Augmentation and Attention for Self-Supervised Monocular Depth EstimationIEEE Robotics and Automation Letters, 2021
- Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset TransferIEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
- Learning Depth from Single Monocular Images Using Deep Convolutional Neural FieldsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015