OctNetFusion: Learning Depth Fusion from Data
Authors:
- Gernot RieglerGRGernot Riegler
- Ali Osman UlusoyAUAli Osman Ulusoy
- Horst BischofHBHorst Bischof
- Andreas GeigerAGAndreas Geiger
- 1 October 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 57-66
Abstract
In this paper, we present a learning based approach to depth fusion, i.e., dense 3D reconstruction from multiple depth images. The most common approach to depth fusion is based on averaging truncated signed distance functions, which was originally proposed by Curless and Levoy in 1996. While this method is simple and provides great results, it is not able to reconstruct (partially) occluded surfaces and requires a large number frames to filter out sensor noise and outliers. Motivated by the availability of large 3D model repositories and recent advances in deep learning, we present a novel 3D CNN architecture that learns to predict an implicit surface representation from the input depth maps. Our learning based method significantly outperforms the traditional volumetric fusion approach in terms of noise reduction and outlier suppression. By learning the structure of real world 3D objects and scenes, our approach is further able to reconstruct occluded regions and to fill in gaps in the reconstruction. We demonstrate that our learning based approach outperforms both vanilla TSDF fusion as well as TV-L1 fusion on the task of volumetric fusion. Further, we demonstrate state-of-the-art 3D shape completion results.Keywords
All Related Versions
This publication has 37 references indexed in Scilit:
- OctNet: Learning Deep 3D Representations at High Resolutions2017
- Volumetric and Multi-view CNNs for Object Classification on 3D Data2016
- Structure-from-Motion Revisited2016
- VoxNet: A 3D Convolutional Neural Network for real-time object recognition2015
- Discrete optimization of ray potentials for semantic 3D reconstruction2015
- Real-time 3D reconstruction at scale using voxel hashingACM Transactions on Graphics, 2013
- High quality depth map upsampling for 3D-TOF cameras2011
- KinectFusion: Real-time dense surface mapping and tracking2011
- Real-time rendering and dynamic updating of 3-d volumetric data2011
- Marching cubes: A high resolution 3D surface construction algorithmACM SIGGRAPH Computer Graphics, 1987