Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements
Authors:
- 1 June 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8170-8179
Abstract
Removing undesirable reflections from a single image captured through a glass window is of practical importance to visual computing systems. Although state-of-the-art methods can obtain decent results in certain situations, performance declines significantly when tackling more general real-world cases. These failures stem from the intrinsic difficulty of single image reflection removal -- the fundamental ill-posedness of the problem, and the insufficiency of densely-labeled training data needed for resolving this ambiguity within learning-based neural network pipelines. In this paper, we address these issues by exploiting targeted network enhancements and the novel use of misaligned data. For the former, we augment a baseline network architecture by embedding context encoding modules that are capable of leveraging high-level contextual clues to reduce indeterminacy within areas containing strong reflections. For the latter, we introduce an alignment-invariant loss function that facilitates exploiting misaligned real-world training data that is much easier to collect. Experimental results collectively show that our method outperforms the state-of-the-art with aligned data, and that significant improvements are possible when using additional misaligned data.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- A computational approach for obstruction-free photographyACM Transactions on Graphics, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- A Physically-Based Approach to Reflection Separation: From Physical Modeling to Constrained OptimizationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Image-based rendering for scenes with reflectionsACM Transactions on Graphics, 2012
- Image smoothing viaL0gradient minimizationACM Transactions on Graphics, 2011
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- Removing photography artifacts using gradient projection and flash-exposure samplingACM Transactions on Graphics, 2005
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Layer extraction from multiple images containing reflections and transparency2002