Bayesian DeNet: Monocular Depth Prediction and Frame-Wise Fusion With Synchronized Uncertainty
- 18 April 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 21 (11) , 2701-2713
Abstract
Using deep convolutional neural networks (CNN) to predict depth from a single image has received considerable attention in recent years due to its impressive performance. However, existing methods process each single image independently without leveraging the multi-view information of video sequences in practical scenarios. Properly taking into account multi-view information in video sequences beyond individual frames could offer considerable benefits in terms of depth prediction accuracy and robustness. In addition, a meaningful measure of prediction uncertainty is essential for decision making, which is not provided in existing methods. This paper presents a novel video-based depth prediction system (DPS) based on a monocular camera, named Bayesian DeNet. Specifically, Bayesian DeNet consists of a 59-layers CNN that can concurrently output a depth map and an uncertainty map for each video frame. Each pixel in an uncertainty map indicates the error variance of the corresponding depth estimate. Depth estimates and uncertainties of previous frames are propagated to the current frame based on the tracked camera pose, yielding multiple depth/uncertainty hypotheses for the current frame which are then fused in a Bayesian inference framework for greater accuracy and robustness. Extensive experiments on three public datasets demonstrate that our Bayesian DeNet outperforms the state-of-the-art methods for monocular depth prediction. A demo video and code are publicly available.Keywords
Funding Information
- National Natural Science Foundation of China (61872417, 61502188)
- Wuhan Science and Technology Bureau (2017010201010111)
- Program for HUST Acadamic Frontier Youth Team
This publication has 35 references indexed in Scilit:
- Learning Depth from Single Monocular Images Using Deep Convolutional Neural FieldsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Deep convolutional neural fields for depth estimation from a single image2015
- Towards unified depth and semantic prediction from a single image2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- An Accurate and Robust Range Image Registration Algorithm for 3D Object ModelingIEEE Transactions on Multimedia, 2014
- Semi-dense Visual Odometry for a Monocular Camera2013
- Vision meets robotics: The KITTI datasetThe International Journal of Robotics Research, 2013
- Indoor Segmentation and Support Inference from RGBD Images2012
- Structure from motion2010
- High speed obstacle avoidance using monocular vision and reinforcement learning2005