LERPS: Lighting Estimation and Relighting for Photometric Stereo
Authors:
- 23 May 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2060-2064
Abstract
Photometric stereo is a method to obtain surface normals of an object using its images captured under varying illumination directions. The existing deep learning-based methods require multiple images of an object captured using complex image acquisition systems. In this work, we propose a deep learning framework to perform three tasks jointly: (i) lighting estimation, (ii) image relighting, and (iii) surface normal estimation, all from a single input image of an object with non-Lambertian surface and general reflectance. The network explicitly segregates global geometric features and local lighting-specific features of the object from a single image. The local features resemble attached shadows, shadings, and specular highlights, providing valuable lighting estimation and relighting cues. The global features capture the lighting-independent geometric attributes that effectively guide the surface normal estimation. The joint training transfers valuable insights to achieve significant improvements across all three tasks. We show that the proposed single-image-based relighting framework outperforms several existing photometric stereo methods which require multiple images of a static object.Keywords
This publication has 29 references indexed in Scilit:
- Deep Residual Learning for Image Recognition2016
- A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo2016
- A Survey of Photometric Stereo TechniquesFoundations and Trends in Computer Graphics and Vision, 2015
- A Closed-Form, Consistent and Robust Solution to Uncalibrated Photometric Stereo Via Local Diffuse Reflectance MaximaInternational Journal of Computer Vision, 2013
- Calibrating Photometric Stereo by Holistic Reflectance Symmetry Analysis2013
- Resolving the Generalized Bas-Relief Ambiguity by Entropy Minimization2007
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- A mathematical theory of communicationACM SIGMOBILE Mobile Computing and Communications Review, 2001
- Acquiring the reflectance field of a human face2000
- Photometric Method For Determining Surface Orientation From Multiple ImagesOptical Engineering, 1980