Video Reflection Removal Through Spatio-Temporal Optimization
Authors:
- Ajay NandoriyaANAjay Nandoriya
- Mohamed ElgharibMEMohamed Elgharib
- Changil KimCKChangil Kim
- Mohamed HefeedaMHMohamed Hefeeda
- Wojciech MatusikWMWojciech Matusik
- 1 October 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2430-2438
Abstract
Reflections can obstruct content during video capture and hence their removal is desirable. Current removal techniques are designed for still images, extracting only one reflection (foreground) and one background layer from the input. When extended to videos, unpleasant artifacts such as temporal flickering and incomplete separation are generated. We present a technique for video reflection removal by jointly solving for motion and separation. The novelty of our work is in our optimization formulation as well as the motion initialization strategy. We present a novel spatiotemporal optimization that takes n frames as input and directly estimates 2n frames as output, n for each layer. We aim to fully utilize spatio-temporal information in our objective terms. Our motion initialization is based on iterative frame-to-frame alignment instead of the direct alignment used by current approaches. We compare against advanced video extensions of the state of the art, and we significantly reduce temporal flickering and improve separation. In addition, we reduce image blur and recover moving objects more accurately. We validate our approach through subjective and objective evaluations on real and controlled data.Keywords
This publication has 22 references indexed in Scilit:
- A computational approach for obstruction-free photographyACM Transactions on Graphics, 2015
- Exploiting Reflection Change for Automatic Reflection Removal2013
- User-assisted reflection detection and feature point tracking2013
- Image-based rendering in the gradient domainACM Transactions on Graphics, 2013
- Image-based rendering for scenes with reflectionsACM Transactions on Graphics, 2012
- Practical temporal consistency for image-based graphics applicationsACM Transactions on Graphics, 2012
- Sparse ICA for blind separation of transmitted and reflected imagesInternational Journal of Imaging Systems and Technology, 2005
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Separating Transparent Layers through Layer Information Exchange2004
- Layer extraction from multiple images containing reflections and transparency2002