Photometric Mesh Optimization for Video-Aligned 3D Object Reconstruction
Authors:
- Vladimir G. KimVKVladimir G. Kim
- 1 June 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 969-978
Abstract
In this paper, we address the problem of 3D object mesh reconstruction from RGB videos. Our approach combines the best of multi-view geometric and data-driven methods for 3D reconstruction by optimizing object meshes for multi-view photometric consistency while constraining mesh deformations with a shape prior. We pose this as a piecewise image alignment problem for each mesh face projection. Our approach allows us to update shape parameters from the photometric error without any depth or mask information. Moreover, we show how to avoid a degeneracy of zero photometric gradients via rasterizing from a virtual viewpoint. We demonstrate 3D object mesh reconstruction results from both synthetic and real-world videos with our photometric mesh optimization, which is unachievable with either naive mesh generation networks or traditional pipelines of surface reconstruction without heavy manual post-processing.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- ORB-SLAM: A Versatile and Accurate Monocular SLAM SystemIEEE Transactions on Robotics, 2015
- Photometric Bundle Adjustment for Dense Multi-view 3D Modeling2014
- Screened poisson surface reconstructionACM Transactions on Graphics, 2013
- DTAM: Dense tracking and mapping in real-time2011
- Gradient Flows for Optimizing Triangular Mesh-based Surfaces: Applications to 3D Reconstruction Problems Dealing with VisibilityInternational Journal of Computer Vision, 2010
- Active Appearance Models RevisitedInternational Journal of Computer Vision, 2004
- Lucas-Kanade 20 Years On: A Unifying FrameworkInternational Journal of Computer Vision, 2004
- Active appearance modelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- Active Shape Models — ‘Smart Snakes’1992
- Animating rotation with quaternion curves1985