Cooperative patch-based 3D surface tracking
Authors:
- Martin KlaudinyMKMartin Klaudiny
- Adrian HiltonAHAdrian Hilton
- 1 November 2011
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 67-76
Abstract
This paper presents a novel dense motion capture technique which creates a temporally consistent mesh sequence from several calibrated and synchronised video sequences of a dynamic object. A surface patch model based on the topology of a user-specified reference mesh is employed to track the surface of the object over time. Multi-view 3D matching of surface patches using a novel cooperative minimisation approach provides initial motion estimates which are robust to large, rapid non-rigid changes of shape. A Laplacian deformation subsequently regularises the motion of the whole mesh using the weighted vertex displacements as soft constraints. An unregistered surface geometry independently reconstructed at each frame is incorporated as a shape prior to improve the quality of tracking. The method is evaluated in a challenging scenario of facial performance capture. Results demonstrate accurate tracking of fast, complex expressions over long sequences without use of markers or a pattern.Keywords
This publication has 20 references indexed in Scilit:
- Multi-scale 3D scene flow from binocular stereo sequencesComputer Vision and Image Understanding, 2008
- A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms2006
- Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching ScoreInternational Journal of Computer Vision, 2006
- Spacetime faces2004
- Laplacian surface editing2004
- Discrete Differential-Geometry Operators for Triangulated 2-Manifolds2003
- Integrated 3D scene flow and structure recovery from multiview image sequences2002
- Spatio-Temporal Stereo Using Multi-Resolution Subdivision SurfacesInternational Journal of Computer Vision, 2002
- Stereo Without Epipolar Lines: A Maximum-Flow FormulationInternational Journal of Computer Vision, 1999
- Three-dimensional scene flow1999