Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF From a Single Image
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2472-2481
Abstract
We propose a deep inverse rendering framework for indoor scenes. From a single RGB image of an arbitrary indoor scene, we obtain a complete scene reconstruction, estimating shape, spatially-varying lighting, and spatially-varying, non-Lambertian surface reflectance. Our novel inverse rendering network incorporates physical insights -- including a spatially-varying spherical Gaussian lighting representation, a differentiable rendering layer to model scene appearance, a cascade structure to iteratively refine the predictions and a bilateral solver for refinement -- allowing us to jointly reason about shape, lighting, and reflectance. Since no existing dataset provides ground truth high quality spatially-varying material and spatially-varying lighting, we propose novel methods to map complex materials to existing indoor scene datasets and a new physically-based GPU renderer to create a large-scale, photorealistic indoor dataset. Experiments show that our framework outperforms previous methods and enables various novel applications like photorealistic object insertion and material editing.Keywords
All Related Versions
This publication has 36 references indexed in Scilit:
- Shape, Illumination, and Reflectance from ShadingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Intrinsic images in the wildACM Transactions on Graphics, 2014
- Anisotropic spherical GaussiansACM Transactions on Graphics, 2013
- Shape and Reflectance from Natural Illumination2012
- Rendering synthetic objects into legacy photographsACM Transactions on Graphics, 2011
- Blind Reflectometry2010
- Efficient Reflectance and Visibility Approximations for Environment Map RenderingComputer Graphics Forum, 2007
- Graphcut texturesACM Transactions on Graphics, 2003
- Real-time texture synthesis by patch-based samplingACM Transactions on Graphics, 2001
- Illumination for computer generated picturesCommunications of the ACM, 1975