Niagara: Normal-Integrated Geometric Affine Field for Scene Reconstruction from a Single View
Authors:
Preprint
Abstract
Recent advances in single-view 3D scene reconstruction have highlighted the challenges in capturing fine geometric details and ensuring structural consistency, particularly in high-fidelity outdoor scene modeling. This paper presents Niagara, a new single-view 3D scene reconstruction framework that can faithfully reconstruct challenging outdoor scenes from a single input image for the first time. Our approach integrates monocular depth and normal estimation as input, which substantially improves its ability to capture fine details, mitigating common issues like geometric detail loss and deformation. Additionally, we introduce a geometric affine field (GAF) and 3D self-attention as geometry-constraint, which combines the structural properties of explicit geometry with the adaptability of implicit feature fields, striking a balance between efficient rendering and high-fidelity reconstruction. Our framework finally proposes a specialized encoder-decoder architecture, where a depth-based 3D Gaussian decoder is proposed to predict 3D Gaussian parameters, which can be used for novel view synthesis. Extensive results and analyses suggest that our Niagara surpasses prior SoTA approaches such as Flash3D in both single-view and dual-view settings, significantly enhancing the geometric accuracy and visual fidelity, especially in outdoor scenes.Keywords
All Related Versions
This publication has 50 references indexed in Scilit:
- LatentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction2024
- MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-view Images2024
- TensoRF: Tensorial Radiance Fields2022
- NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors2022
- R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis2022
- SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentation for Accurate Freespace Detection2020
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- UAV for 3D mapping applications: a reviewApplied Geomatics, 2013
- 3-D Depth Reconstruction from a Single Still ImageInternational Journal of Computer Vision, 2007
- Monocular depth perception from optical flow by space time signal processingProceedings of the Royal Society of London. B. Biological Sciences, 1983