Eclipse: Disambiguating Illumination and Materials Using Unintended Shadows
Authors:
- 16 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 77-86
Abstract
Decomposing an object's appearance into representations of its materials and the surrounding illumination is difficult, even when the object's 3D shape is known before-hand. This problem is especially challenging for diffuse objects: it is ill-conditioned because diffuse materials severely blur incoming light, and it is ill-posed because diffuse materials under high-frequency lighting can be indistinguishable from shiny materials under low-frequency lighting. We show that it is possible to recover precise materials and illumination-even from diffuse objects-by exploiting un-intended shadows, like the ones cast onto an object by the photographer who moves around it. These shadows are a nuisance in most previous inverse rendering pipelines, but here we exploit them as signals that improve conditioning and help resolve material-lighting ambiguities. We present a method based on differentiable Monte Carlo ray tracing that uses images of an object to jointly recover its spatially-varying materials, the surrounding illumination environment, and the shapes of the unseen light occluders who in-advertently cast shadows upon it.Keywords
All Related Versions
This publication has 36 references indexed in Scilit:
- Deep Reflectance Volumes: Relightable Reconstructions from Multi-view Photometric Images2020
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- Seeing the World in a Bag of Chips2020
- Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF From a Single Image2020
- Neural Illumination: Lighting Prediction for Indoor Environments2019
- Inferring Light Fields from Shadows2018
- Deep outdoor illumination estimation2017
- Accidental pinhole and pinspeck cameras: Revealing the scene outside the picture2012
- Estimating the Natural Illumination Conditions from a Single Outdoor ImageInternational Journal of Computer Vision, 2011
- Computing Fourier Transforms and Convolutions on the 2-SphereAdvances in Applied Mathematics, 1994