Differentiable Light Transport with Gaussian Surfels via Adapted Radiosity for Efficient Relighting and Geometry Reconstruction
Open Access
- 4 December 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 44 (6) , 1-25
Abstract
Radiance fields have gained tremendous success with applications ranging from novel view synthesis to geometry reconstruction, especially with the advent of Gaussian splatting. However, they sacrifice modeling of material reflective properties and lighting conditions, leading to significant geometric ambiguities and the inability to easily perform relighting. One way to address these limitations is to incorporate physically-based rendering, but it has been prohibitively expensive to include full global illumination within the inner loop of the optimization. Therefore, previous works adopt simplifications that make the whole optimization with global illumination effects efficient but less accurate. In this work, we adopt Gaussian surfels as the primitives and build an efficient framework for differentiable light transport, inspired from the classic radiosity theory. The whole framework operates in the coefficient space of spherical harmonics, enabling both diffuse and specular materials. We extend the classic radiosity into non-binary visibility and semi-opaque primitives, propose novel solvers to efficiently solve the light transport, and derive the backward pass for gradient optimizations, which is more efficient than auto-differentiation. During inference, we achieve view-independent rendering where light transport need not be recomputed under viewpoint changes, enabling hundreds of FPS for global illumination effects, including view-dependent reflections using a spherical harmonics representation. Through extensive qualitative and quantitative experiments, we demonstrate superior geometry reconstruction, view synthesis and relighting than previous inverse rendering baselines, or data-driven baselines given relatively sparse datasets with known or unknown lighting conditions.Keywords
All Related Versions
Funding Information
- Intelligence Advanced Research Projects Activity (140D0423C0076)
- Office of Naval Research (N00014-23-1-2526)
- NSF (2238839, 2100237, 2120019)
This publication has 90 references indexed in Scilit:
- SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views2022
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- Inverse Path Tracing for Joint Material and Lighting Estimation2019
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- An Evaluation of Computational Imaging Techniques for Heterogeneous Inverse Scattering2016
- Pixelwise View Selection for Unstructured Multi-View Stereo2016
- Discontinuity meshing for accurate radiosityIEEE Computer Graphics and Applications, 1992
- Shape from interreflectionsInternational Journal of Computer Vision, 1991
- Estimation of illuminant direction, albedo, and shape from shadingIEEE Transactions on Pattern Analysis and Machine Intelligence, 1991
- A general purpose unequal probability sampling planBiometrika, 1982