SN2eRF: A Framework for Neural Radiance Fields Given Sparse and Noisy Poses
- 6 August 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 31 (9) , 5188-5198
Abstract
Neural Radiance Fields (NeRFs) have shown impressive capabilities in synthesizing photorealistic novel views. However, their application to room-size scenes is limited by the requirement of several hundred views with accurate poses for training. To address this challenge, we propose SN$^{2}$ eRF, a framework which can reconstruct the neural radiance field with significantly fewer views and noisy poses by exploiting multiple priors. Our key insight is to leverage both multi-view and monocular priors to constrain the optimization of NeRF in the setting of sparse and noisy pose inputs. Specifically, we extract and match key points to constrain pose optimization and use Ray Transformer with a monocular depth estimator to provide dense depth prior for geometry optimization. Benefiting from these priors, our approach achieves state-of-the-art accuracy in novel view synthesis for indoor room scenarios.
Keywords
Funding Information
- National Science and Technology Major Project (2021ZD0112902)
- National Natural Science Foundation of China (62220106003)
- Research Grant of Beijing Higher Institution Engineering Research Center and Tsinghua-Tencent Joint Laboratory for Internet Innovation Technology
This publication has 43 references indexed in Scilit:
- NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM2024
- SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views2022
- Generalizable Patch-Based Neural Rendering2022
- GeoAug: Data Augmentation for Few-Shot NeRF with Geometry Constraints2022
- SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image2022
- MVSTER: Epipolar Transformer for Efficient Multi-view Stereo2022
- ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes2017
- Structure-from-Motion Revisited2016
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- The Elements of Statistical Learning2001