Neural 3D reconstruction from sparse views using geometric priors
Open Access
- 1 December 2023
- journal article
- research article
- Published by Tsinghua University Press in Computational Visual Media
- Vol. 9 (4) , 687-697
Abstract
Sparse view 3D reconstruction has attracted increasing attention with the development of neural implicit 3D representation. Existing methods usually only make use of 2D views, requiring a dense set of input views for accurate 3D reconstruction. In this paper, we show that accurate 3D reconstruction can be achieved by incorporating geometric priors into neural implicit 3D reconstruction. Our method adopts the signed distance function as the 3D representation, and learns a generalizable 3D surface reconstruction model from sparse views. Specifically, we build a more effective and sparse feature volume from the input views by using corresponding depth maps, which can be provided by depth sensors or directly predicted from the input views. We recover better geometric details by imposing both depth and surface normal constraints in addition to the color loss when training the neural implicit 3D representation. Experiments demonstrate that our method both outperforms state-of-the-art approaches, and achieves good generalizability.Keywords
This publication has 46 references indexed in Scilit:
- MVSNet: Depth Inference for Unstructured Multi-view Stereo2018
- Feature Pyramid Networks for Object Detection2017
- A survey of structure from motion.Acta Numerica, 2017
- 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction2016
- Structure-from-Motion Revisited2016
- Large Scale Multi-view Stereopsis Evaluation2014
- Visual simultaneous localization and mapping: a surveyArtificial Intelligence Review, 2012
- Accurate, Dense, and Robust Multiview StereopsisIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009
- A quasi-dense approach to surface reconstruction from uncalibrated imagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2005
- Photorealistic Scene Reconstruction by Voxel ColoringInternational Journal of Computer Vision, 1999