GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces
Authors:
Top Cited Papers
- 16 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5322-5332
Abstract
The advent of neural 3D Gaussians [21] has recently brought about a revolution in the field of neural rendering, facilitating the generation of high-quality renderings at real-time speeds. However, the explicit and discrete repre-sentation encounters challenges when applied to scenes fea-turing reflective surfaces. In this paper, we present Gaus-sian Shader, a novel method that applies a simplified shading function on 3D Gaussians to enhance the neural ren-dering in scenes with reflective surfaces while preserving the training and rendering efficiency. The main challenge in applying the shading function lies in the accurate nor-mal estimation on discrete 3D Gaussians. Specifically, we proposed a novel normal estimation framework based on the shortest axis directions of 3D Gaussians with a deli-cately designed loss to make the consistency between the normals and the geometries of Gaussian spheres. Experiments show that GaussianShader strikes a commendable balance between efficiency and visual quality. Our method surpasses Gaussian Splatting [21] in PSNR on specular object datasets, exhibiting an improvement of 1.57dB. When compared to prior works handling reflective surfaces, such as Ref-NeRF [45], our optimization time is significantly accelerated (23h vs. 0.58h). Please click on our project web-site to see more resultsKeywords
All Related Versions
Funding Information
- NSFC (62206173)
This publication has 42 references indexed in Scilit:
- TensoRF: Tensorial Radiance Fields2022
- NeILF: Neural Incident Light Field for Physically-based Material Estimation2022
- Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction2022
- Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields2022
- Point-NeRF: Point-based Neural Radiance Fields2022
- NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis2021
- PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting2021
- Neural Point-Based Graphics2020
- Seeing the World in a Bag of Chips2020
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018