Intrinsic video and applications
Authors:
Open Access
- 27 July 2014
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 33 (4) , 1-11
Abstract
We present a method to decompose a video into its intrinsic components of reflectance and shading, plus a number of related example applications in video editing such as segmentation, stylization, material editing, recolorization and color transfer. Intrinsic decomposition is an ill-posed problem, which becomes even more challenging in the case of video due to the need for temporal coherence and the potentially large memory requirements of a global approach. Additionally, user interaction should be kept to a minimum in order to ensure efficiency. We propose a probabilistic approach, formulating a Bayesian Maximum a Posteriori problem to drive the propagation of clustered reflectance values from the first frame, and defining additional constraints as priors on the reflectance and shading. We explicitly leverage temporal information in the video by building a causal-anticausal, coarse-to-fine iterative scheme, and by relying on optical flow information. We impose no restrictions on the input video, and show examples representing a varied range of difficult cases. Our method is the first one designed explicitly for video; moreover, it naturally ensures temporal consistency, and compares favorably against the state of the art in this regard.Keywords
Funding Information
- Gobierno de Aragón
- National Natural Science Foundation of China (61327902, 61035002, 61120106003)
- Ministry of Science and Technology of the People's Republic of China (2013AA01A604)
- Ministerio de Economía y Competitividad (TIN2010-21543)
- Beijing Key Laboratory of Multi-dimensional & Multi-scale Computational Photography
- Seventh Framework Programme
This publication has 29 references indexed in Scilit:
- Tonal stabilization of videoACM Transactions on Graphics, 2011
- GradientShopACM Transactions on Graphics, 2010
- User-assisted intrinsic imagesACM Transactions on Graphics, 2009
- Efficient affinity-based edit propagation using K-D treeACM Transactions on Graphics, 2009
- Video SnapCutACM Transactions on Graphics, 2009
- Intrinsic colorizationACM Transactions on Graphics, 2008
- Factored time-lapse videoACM Transactions on Graphics, 2007
- Illumination normalization with time-dependent intrinsic images for video surveillanceIEEE Transactions on Pattern Analysis and Machine Intelligence, 2004
- Poisson image editingACM Transactions on Graphics, 2003
- Lightness and Retinex TheoryJournal of the Optical Society of America, 1971