Spatially-Adaptive Image Restoration using Distortion-Guided Networks
Authors:
- 1 October 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2289-2299
Abstract
We present a general learning-based solution for restoring images suffering from spatially-varying degradations. Prior approaches are typically degradation-specific and employ the same processing across different images and different pixels within. However, we hypothesize that such spatially rigid processing is suboptimal for simultaneously restoring the degraded pixels as well as reconstructing the clean regions of the image. To overcome this limitation, we propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts computation to difficult regions in the image. SPAIR comprises of two components, (1) a localization network that identifies degraded pixels, and (2) a restoration network that exploits knowledge from the localization network in filter and feature domain to selectively and adaptively restore degraded pixels. Our key idea is to exploit the non-uniformity of heavy degradations in spatial-domain and suitably embed this knowledge within distortion-guided modules performing sparse normalization, feature extraction and attention. Our architecture is agnostic to physical formation model and generalizes across several types of spatially-varying degradations. We demonstrate the efficacy of SPAIR individually on four restoration tasks- removal of rain-streaks, raindrops, shadows and motion blur. Extensive qualitative and quantitative comparisons with prior art on 11 benchmark datasets demonstrate that our degradation-agnostic network design offers significant performance gains over state-of-the-art degradation-specific architectures. Code available at https://github.com/humananalysis/spatially-adaptive-image-restoration.Keywords
All Related Versions
This publication has 55 references indexed in Scilit:
- Rain drop detection and removal using K-Means clustering2015
- Removing Rain from a Single Image via Discriminative Sparse Coding2015
- Automatic Shadow Detection and Removal from a Single ImageIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Single image rain and snow removal via guided L0 smoothing filterMultimedia Tools and Applications, 2015
- Learning a convolutional neural network for non-uniform motion blur removal2015
- Interactive Shadow Removal and Ground Truth for Variable Scene Categories2014
- Restoring an Image Taken through a Window Covered with Dirt or Rain2013
- Unnatural L0 Sparse Representation for Natural Image Deblurring2013
- Paired Regions for Shadow Detection and RemovalIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Shadow Removal Using Bilateral FilteringIEEE Transactions on Image Processing, 2012