PlaNeRF: SVD Unsupervised 3D Plane Regularization for NeRF Large-Scale Scene Reconstruction
Preprint
Abstract
Neural Radiance Fields (NeRF) enable 3D scene reconstruction from 2D images and camera poses for Novel View Synthesis (NVS). Although NeRF can produce photorealistic results, it often suffers from overfitting to training views, leading to poor geometry reconstruction, especially in low-texture areas. This limitation restricts many important applications which require accurate geometry, such as extrapolated NVS, HD mapping and scene editing. To address this limitation, we propose a new method to improve NeRF's 3D structure using only RGB images and semantic maps. Our approach introduces a novel plane regularization based on Singular Value Decomposition (SVD), that does not rely on any geometric prior. In addition, we leverage the Structural Similarity Index Measure (SSIM) in our loss design to properly initialize the volumetric representation of NeRF. Quantitative and qualitative results show that our method outperforms popular regularization approaches in accurate geometry reconstruction for large-scale outdoor scenes and achieves SoTA rendering quality on the KITTI-360 NVS benchmark.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- MARS: An Instance-Aware, Modular and Realistic Simulator for Autonomous Driving2024
- MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal Calibration2023
- StructNeRF: Neural Radiance Fields for Indoor Scenes With Structural HintsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- FreeNeRF: Improving Few-Shot Neural Rendering with Free Frequency Regularization2023
- Nerflets: Local Radiance Fields for Efficient Structure-Aware 3D Scene Representation from 2D Supervision2023
- Neural 3D Reconstruction in the Wild2022
- Instant neural graphics primitives with a multiresolution hash encodingACM Transactions on Graphics, 2022
- KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3DIEEE Transactions on Pattern Analysis and Machine Intelligence, 2022
- Global Stereo Reconstruction under Second-Order Smoothness PriorsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009