Single image portrait relighting
Authors:
Open Access
- 12 July 2019
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 38 (4) , 1-12
Abstract
Lighting plays a central role in conveying the essence and depth of the subject in a portrait photograph. Professional photographers will carefully control the lighting in their studio to manipulate the appearance of their subject, while consumer photographers are usually constrained to the illumination of their environment. Though prior works have explored techniques for relighting an image, their utility is usually limited due to requirements of specialized hardware, multiple images of the subject under controlled or known illuminations, or accurate models of geometry and reflectance. To this end, we present a system for portrait relighting : a neural network that takes as input a single RGB image of a portrait taken with a standard cellphone camera in an unconstrained environment, and from that image produces a relit image of that subject as though it were illuminated according to any provided environment map. Our method is trained on a small database of 18 individuals captured under different directional light sources in a controlled light stage setup consisting of a densely sampled sphere of lights. Our proposed technique produces quantitatively superior results on our dataset's validation set compared to prior works, and produces convincing qualitative relighting results on a dataset of hundreds of real-world cellphone portraits. Because our technique can produce a 640 × 640 image in only 160 milliseconds, it may enable interactive user-facing photographic applications in the future.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Skin microstructure deformation with displacement map convolutionACM Transactions on Graphics, 2015
- Style transfer for headshot portraitsACM Transactions on Graphics, 2014
- Make3D: Learning 3D Scene Structure from a Single Still ImageIEEE Transactions on Pattern Analysis and Machine Intelligence, 2008
- Recovering Surface Layout from an ImageInternational Journal of Computer Vision, 2007
- Performance relighting and reflectance transformation with time-multiplexed illuminationACM Transactions on Graphics, 2005
- Digital photography with flash and no-flash image pairs2004
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Acquiring the reflectance field of a human face2000
- A morphable model for the synthesis of 3D faces1999
- Backpropagation Applied to Handwritten Zip Code RecognitionNeural Computation, 1989