SDC-Depth: Semantic Divide-and-Conquer Network for Monocular Depth Estimation
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 538-547
Abstract
Monocular depth estimation is an ill-posed problem, and as such critically relies on scene priors and semantics. Due to its complexity, we propose a deep neural network model based on a semantic divide-and-conquer approach. Our model decomposes a scene into semantic segments, such as object instances and background stuff classes, and then predicts a scale and shift invariant depth map for each semantic segment in a canonical space. Semantic segments of the same category share the same depth decoder, so the global depth prediction task is decomposed into a series of category-specific ones, which are simpler to learn and easier to generalize to new scene types. Finally, our model stitches each local depth segment by predicting its scale and shift based on the global context of the image. The model is trained end-to-end using a multi-task loss for panoptic segmentation and depth prediction, and is therefore able to leverage large-scale panoptic segmentation datasets to boost its semantic understanding. We validate the effectiveness of our approach and show state-of-the-art performance on three benchmark datasets.Keywords
This publication has 30 references indexed in Scilit:
- Unsupervised Monocular Depth Estimation with Left-Right Consistency2017
- Unsupervised Learning of Depth and Ego-Motion from Video2017
- Deeper Depth Prediction with Fully Convolutional Residual Networks2016
- Deep Residual Learning for Image Recognition2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Monocular Depth Estimation Using Neural Regression Forest2016
- Deep convolutional neural fields for depth estimation from a single image2015
- Towards unified depth and semantic prediction from a single image2015
- Fully convolutional networks for semantic segmentation2015
- Make3D: Learning 3D Scene Structure from a Single Still ImageIEEE Transactions on Pattern Analysis and Machine Intelligence, 2008