Patch-Based Video Processing: A Variational Bayesian Approach
Authors:
- 23 September 2008
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 19 (1) , 27-40
Abstract
In this paper, we present a patch-based variational Bayesian framework for video processing and demonstrate its potential in denoising, inpainting and deinterlacing. Unlike previous methods based on explicit motion estimation, we propose to embed motion-related information into the relationship among video patches and develop a nonlocal sparsity-based prior for typical video sequences. Specifically, we first extend block matching (nearest neighbor search) into patch clustering (k-nearest-neighbor search), which represents motion in an implicit and distributed fashion. Then we show how to exploit the sparsity constraint by sorting and packing similar patches, which can be better understood from a manifold perspective. Under the Bayesian framework, we treat both patch clustering result and unobservable data as latent variables and solve the inference problem via variational EM algorithms. A weighted averaging strategy of fusing diverse inference results from overlapped patches is also developed. The effectiveness of patch-based video models is demonstrated by extensive experimental results on a wide range of video materials.Keywords
This publication has 71 references indexed in Scilit:
- Weighted Averaging for Denoising With Overcomplete DictionariesIEEE Transactions on Image Processing, 2007
- An Inpainting- Based Deinterlacing MethodIEEE Transactions on Image Processing, 2007
- Video Inpainting Under Constrained Camera MotionIEEE Transactions on Image Processing, 2007
- Video de-interlacing by adaptive 4-field global/local motion compensated approachIEEE Transactions on Circuits and Systems for Video Technology, 2005
- Motion Compensation Assisted Motion Adaptive Interlaced-to-Progressive ConversionIEEE Transactions on Circuits and Systems for Video Technology, 2004
- Reliable motion detection/compensation for interlaced sequences and its applications to deinterlacingIEEE Transactions on Circuits and Systems for Video Technology, 2000
- Deinterlacing-an overviewProceedings of the IEEE, 1998
- Deinterlacing by successive approximationIEEE Transactions on Image Processing, 1997
- Digitally assisted deinterlacing for, EDTVIEEE Transactions on Circuits and Systems for Video Technology, 1993
- Adaptive Mixtures of Local ExpertsNeural Computation, 1991