Motion layer extraction in the presence of occlusion using graph cuts
Authors:
- 22 August 2005
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 27 (10) , 1644-1659
Abstract
Extracting layers from video is very important for video representation, analysis, compression, and synthesis. Assuming that a scene can be approximately described by multiple planar regions, this paper describes a robust and novel approach to automatically extract a set of affine or projective transformations induced by these regions, detect the occlusion pixels over multiple consecutive frames, and segment the scene into several motion layers. First, after determining a number of seed regions using correspondences in two frames, we expand the seed regions and reject the outliers employing the graph cuts method integrated with level set representation. Next, these initial regions are merged into several initial layers according to the motion similarity. Third, an occlusion order constraint on multiple frames is explored, which enforces that the occlusion area increases with the temporal order in a short period and effectively maintains segmentation consistency over multiple consecutive frames. Then, the correct layer segmentation is obtained by using a graph cuts algorithm and the occlusions between the overlapping layers are explicitly determined. Several experimental results are demonstrated to show that our approach is effective and robust.Keywords
This publication has 32 references indexed in Scilit:
- Object segmentation using graph cuts based active contoursComputer Vision and Image Understanding, 2007
- Object based segmentation of video using color, motion and spatial information2005
- A subspace approach to layer extraction2005
- Graphcut textures2003
- Level Set Methods and Dynamic Implicit Surfaces2003
- Two-frame wide baseline matching2003
- A background layer model for object tracking through occlusion2003
- Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding2002
- Video segmentation by MAP labeling of watershed segmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving ObjectsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1985