A Novel Method for Monocular Depth Estimation Using an Hourglass Neck Module
Open Access
Abstract
In this paper, we propose a novel method for monocular depth estimation using the hourglass neck module. The proposed method has the following originality. First, feature maps are extracted from Swin Transformer V2 using a masked image modeling (MIM) pretrained model. Since Swin Transformer V2 has a different patch size for each attention stage, it is easier to extract local and global features from images input by the vision transformer (ViT)-based encoder. Second, to maintain the polymorphism and local inductive bias of the feature map extracted from Swin Transformer V2, a feature map is input into the hourglass neck module. Third, deformable attention can be used at the waist of the hourglass neck module to reduce the computation cost and highlight the locality of the feature map. Finally, the feature map traverses the neck and proceeds through a decoder, comprised of a deconvolution layer and an upsampling layer, to generate a depth image. To evaluate the objective reliability of the proposed method in this paper, we used the NYU Depth V2 dataset to compare and evaluate the methods published in other papers. As a result of the experiment, the RMSE value of the novel method for monocular depth estimation using the hourglass neck module proposed in this paper was 0.274, which was lower than those published in other papers. The lower the RMSE value, the better the depth estimation method; therefore, its efficiency compared to other techniques has been proven.Keywords
Funding Information
- National Research Foundation of Korea (NRF-2019R1F1A1063984, 2021RIS-004, IITP-2022-RS-2022-00156212, 5199990813896)
- “Regional Innovation Strategy (RIS)” through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (MOE) (NRF-2019R1F1A1063984, 2021RIS-004, IITP-2022-RS-2022-00156212, 5199990813896)
- MSIT (Ministry of Science and ICT), Korea, under the ICAN (ICT Challenge and Advanced Network of HRD) program (NRF-2019R1F1A1063984, 2021RIS-004, IITP-2022-RS-2022-00156212, 5199990813896)
- BK21 Program funded by the Ministry of Education and National Research Foundation of Korea (NRF-2019R1F1A1063984, 2021RIS-004, IITP-2022-RS-2022-00156212, 5199990813896)
This publication has 14 references indexed in Scilit:
- SimMIM: a Simple Framework for Masked Image Modeling2022
- Vision Transformers for Dense Prediction2021
- A Novel Method for Estimating Monocular Depth Using Cycle GAN and SegmentationSensors, 2020
- Pyramid Stereo Matching Network2018
- The curse(s) of dimensionalityNature Methods, 2018
- Unsupervised Learning of Stereo Matching2017
- Unsupervised Monocular Depth Estimation with Left-Right Consistency2017
- ImageNet classification with deep convolutional neural networksCommunications of the ACM, 2017
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Backpropagation Applied to Handwritten Zip Code RecognitionNeural Computation, 1989