A Variational Approach to Video Registration with Subspace Constraints
Open Access
- 2 April 2013
- journal article
- research article
- Published by Springer Nature in International Journal of Computer Vision
- Vol. 104 (3) , 286-314
Abstract
This paper addresses the problem of non-rigid video registration, or the computation of optical flow from a reference frame to each of the subsequent images in a sequence, when the camera views deformable objects. We exploit the high correlation between 2D trajectories of different points on the same non-rigid surface by assuming that the displacement of any point throughout the sequence can be expressed in a compact way as a linear combination of a low-rank motion basis. This subspace constraint effectively acts as a trajectory regularization term leading to temporally consistent optical flow. We formulate it as a robust soft constraint within a variational framework by penalizing flow fields that lie outside the low-rank manifold. The resulting energy functional can be decoupled into the optimization of the brightness constancy and spatial regularization terms, leading to an efficient optimization scheme. Additionally, we propose a novel optimization scheme for the case of vector valued images, based on the dualization of the data term. This allows us to extend our approach to deal with colour images which results in significant improvements on the registration results. Finally, we provide a new benchmark dataset, based on motion capture data of a flag waving in the wind, with dense ground truth optical flow for evaluation of multi-frame optical flow algorithms for non-rigid surfaces. Our experiments show that our proposed approach outperforms state of the art optical flow and dense non-rigid registration algorithms.Keywords
This publication has 45 references indexed in Scilit:
- A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging ScienceSIAM Journal on Imaging Sciences, 2010
- Global Solutions of Variational Models with Convex RegularizationSIAM Journal on Imaging Sciences, 2010
- Capturing and animating occluded clothACM Transactions on Graphics, 2007
- Image Sharpening by Flows Based on Triple Well PotentialsJournal of Mathematical Imaging and Vision, 2004
- Computing Optical Flow via Variational TechniquesSIAM Journal on Applied Mathematics, 1999
- Color TV: total variation methods for restoration of vector-valued imagesIEEE Transactions on Image Processing, 1998
- Color SnakesComputer Vision and Image Understanding, 1997
- Nonlinear total variation based noise removal algorithmsPhysica D: Nonlinear Phenomena, 1992
- A computational approach to motion perceptionBiological Cybernetics, 1988
- Determining optical flowArtificial Intelligence, 1981