MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6108-6118
Abstract
In this paper, we propose MonoRec, a semi-supervised monocular dense reconstruction architecture that predicts depth maps from a single moving camera in dynamic environments. MonoRec is based on a multi-view stereo setting which encodes the information of multiple consecutive images in a cost volume. To deal with dynamic objects in the scene, we introduce a MaskModule that predicts moving object masks by leveraging the photometric inconsistencies encoded in the cost volumes. Unlike other multi-view stereo methods, MonoRec is able to reconstruct both static and moving objects by leveraging the predicted masks. Furthermore, we present a novel multi-stage training scheme with a semi-supervised loss formulation that does not require LiDAR depth values. We carefully evaluate MonoRec on the KITTI dataset and show that it achieves state-of-theart performance compared to both multi-view and singleview methods. With the model trained on KITTI, we furthermore demonstrate that MonoRec is able to generalize well to both the Oxford RobotCar dataset and the more challenging TUM-Mono dataset recorded by a handheld camera. Code and related materials are available at https://vision.in.tum.de/research/monorec.Keywords
All Related Versions
This publication has 46 references indexed in Scilit:
- 1 year, 1000 km: The Oxford RobotCar datasetThe International Journal of Robotics Research, 2016
- Deeper Depth Prediction with Fully Convolutional Residual Networks2016
- Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture2015
- Massively Parallel Multiview Stereopsis by Surface Normal Diffusion2015
- DTAM: Dense tracking and mapping in real-time2011
- Real-Time Dense Geometry from a Handheld Camera2010
- Accurate, Dense, and Robust Multiview StereopsisIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009
- A quasi-dense approach to surface reconstruction from uncalibrated imagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2005
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Photorealistic scene reconstruction by voxel coloring2002