Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Rendering
Authors:
- 15 July 2021
- journal article
- conference paper
- Published by Wiley in Computer Graphics Forum
- Vol. 40 (4) , 101-113
Abstract
Reconstructing the shape and appearance of real‐world objects using measured 2D images has been a long‐standing inverse rendering problem. In this paper, we introduce a new analysis‐by‐synthesis technique capable of producing high‐quality reconstructions through robust coarse‐to‐fine optimization and physics‐based differentiable rendering.Unlike most previous methods that handle geometry and reflectance largely separately, our method unifies the optimization of both by leveraging image gradients with respect to both object reflectance and geometry. To obtain physically accurate gradient estimates, we develop a new GPU‐based Monte Carlo differentiable renderer leveraging recent advances in differentiable rendering theory to offer unbiased gradients while enjoying better performance than existing tools like PyTorch3D [RRN*20] and redner [LADL18]. To further improve robustness, we utilize several shape and material priors as well as a coarse‐to‐fine optimization strategy to reconstruct geometry. Using both synthetic and real input images, we demonstrate that our technique can produce reconstructions with higher quality than previous methods.Keywords
All Related Versions
This publication has 63 references indexed in Scilit:
- Pocket reflectometryACM Transactions on Graphics, 2011
- A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectanceACM Transactions on Graphics, 2010
- Manifold bootstrapping for SVBRDF captureACM Transactions on Graphics, 2010
- Coarse-to-fine surface reconstruction from silhouettes and range data using mesh deformationComputer Vision and Image Understanding, 2010
- A photometric approach for estimating normals and tangentsACM Transactions on Graphics, 2008
- Practical modeling and acquisition of layered facial reflectanceACM Transactions on Graphics, 2008
- Competing Fronts for Coarse–to–Fine Surface ReconstructionComputer Graphics Forum, 2006
- Image-based reconstruction of spatial appearance and geometric detailACM Transactions on Graphics, 2003
- Numerical shape from shading and occluding boundariesArtificial Intelligence, 1981
- Photometric Method For Determining Surface Orientation From Multiple ImagesOptical Engineering, 1980