CoR-GS: Sparse-View 3D Gaussian Splatting via Co-Regularization
Authors:
Preprint
Abstract
3D Gaussian Splatting (3DGS) creates a radiance field consisting of 3D Gaussians to represent a scene. With sparse training views, 3DGS easily suffers from overfitting, negatively impacting the reconstruction quality. This paper introduces a new co-regularization perspective for improving sparse-view 3DGS. When training two 3D Gaussian radiance fields with the same sparse views of a scene, we observe that the two radiance fields exhibit \textit{point disagreement} and \textit{rendering disagreement} that can unsupervisedly predict reconstruction quality, stemming from the sampling implementation in densification. We further quantify the point disagreement and rendering disagreement by evaluating the registration between Gaussians' point representations and calculating differences in their rendered pixels. The empirical study demonstrates the negative correlation between the two disagreements and accurate reconstruction, which allows us to identify inaccurate reconstruction without accessing ground-truth information. Based on the study, we propose CoR-GS, which identifies and suppresses inaccurate reconstruction based on the two disagreements: (\romannumeral1) Co-pruning considers Gaussians that exhibit high point disagreement in inaccurate positions and prunes them. (\romannumeral2) Pseudo-view co-regularization considers pixels that exhibit high rendering disagreement are inaccurately rendered and suppress the disagreement. Results on LLFF, Mip-NeRF360, DTU, and Blender demonstrate that CoR-GS effectively regularizes the scene geometry, reconstructs the compact representations, and achieves state-of-the-art novel view synthesis quality under sparse training views.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- FSGS: Real-Time Few-Shot View Synthesis Using Gaussian Splatting2024
- DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization2024
- 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering2024
- Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis2024
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingACM Transactions on Graphics, 2023
- SparseFusion: Distilling View-Conditioned Diffusion for 3D Reconstruction2023
- Instant neural graphics primitives with a multiresolution hash encodingACM Transactions on Graphics, 2022
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004