Multiview Intrinsic Images of Outdoors Scenes with an Application to Relighting
Authors:
Open Access
- 3 November 2015
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 34 (5) , 1-16
Abstract
We introduce a method to compute intrinsic images for a multiview set of outdoor photos with cast shadows, taken under the same lighting. We use an automatic 3D reconstruction from these photos and the sun direction as input and decompose each image into reflectance and shading layers, despite the inaccuracies and missing data of the 3D model. Our approach is based on two key ideas. First, we progressively improve the accuracy of the parameters of our image formation model by performing iterative estimation and combining 3D lighting simulation with 2D image optimization methods. Second, we use the image formation model to express reflectance as a function of discrete visibility values for shadow and light, which allows to introduce a robust visibility classifier for pairs of points in a scene. This classifier is used for shadow labeling, allowing to compute high-quality reflectance and shading layers. Our multiview intrinsic decomposition is of sufficient quality to allow relighting of the input images. We create shadow-caster geometry which preserves shadow silhouettes and, using the intrinsic layers, we can perform multiview relighting with moving cast shadows. We present results on several multiview datasets, and show how it is now possible to perform image-based rendering with changing illumination conditions.Keywords
Funding Information
- European Commission (VERVE (www.verveconsortium.eu) and CR-PLAY (www.cr-play.eu))
This publication has 45 references indexed in Scilit:
- Ambient point clouds for view interpolationACM Transactions on Graphics, 2010
- Webcam clip artACM Transactions on Graphics, 2009
- User-assisted intrinsic imagesACM Transactions on Graphics, 2009
- SkyFinderACM Transactions on Graphics, 2009
- The Shadow Meets the Mask: Pyramid‐Based Shadow RemovalComputer Graphics Forum, 2008
- Factored time-lapse videoACM Transactions on Graphics, 2007
- Natural shadow mattingACM Transactions on Graphics, 2007
- Photo tourismACM Transactions on Graphics, 2006
- Classification of Illumination Methods for Mixed RealityComputer Graphics Forum, 2006
- Lightness and Retinex TheoryJournal of the Optical Society of America, 1971