Two-Shot Spatially-Varying BRDF and Shape Estimation
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3981-3990
Abstract
Capturing the shape and spatially-varying appearance (SVBRDF) of an object from images is a challenging task that has applications in both computer vision and graphics. Traditional optimization-based approaches often need a large number of images taken from multiple views in a controlled environment. Newer deep learning-based approaches require only a few input images, but the reconstruction quality is not on par with optimization techniques. We propose a novel deep learning architecture with a stage-wise estimation of shape and SVBRDF. The previous predictions guide each estimation, and a joint refinement network later refines both SVBRDF and shape. We follow a practical mobile image capture setting and use unaligned two-shot flash and no-flash images as input. Both our two-shot image capture and network inference can run on mobile hardware. We also create a large-scale synthetic training dataset with domain-randomized geometry and realistic materials. Extensive experiments on both synthetic and real-world datasets show that our network trained on a synthetic dataset can generalize well to real-world images. Comparisons with recent approaches demonstrate the superior performance of the proposed approach.Keywords
All Related Versions
This publication has 48 references indexed in Scilit:
- Practical SVBRDF capture in the frequency domainACM Transactions on Graphics, 2013
- Manifold bootstrapping for SVBRDF captureACM Transactions on Graphics, 2010
- All-frequency rendering of dynamic, spatially-varying reflectanceACM Transactions on Graphics, 2009
- "GrabCut"ACM Transactions on Graphics, 2004
- Planned Sampling of Spatially Varying BRDFsComputer Graphics Forum, 2003
- Image-Based Reconstruction of Spatially Varying Materials2001
- Shape-from-shading: a surveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 1999
- The rendering equationACM SIGGRAPH Computer Graphics, 1986
- A Reflectance Model for Computer GraphicsACM Transactions on Graphics, 1982
- Directional Reflectance and Emissivity of an Opaque SurfaceApplied Optics, 1965