Attentive Feature Refinement Network for Single Rainy Image Restoration
- 17 March 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 30 (10577149) , 3734-3747
Abstract
Despite the fact that great progress has been made on single image deraining tasks, it is still challenging for existing models to produce satisfactory results directly, and it often requires a single or multiple refinement stages to gradually improve the quality. However, in this paper, we demonstrate that existing image-level refinement with a stage-independent learning design is problematic with the side effect of over/under-deraining. To resolve this issue, we for the first time propose the mechanism of learning to carry out refinement on the unsatisfactory features, and propose a novel attentive feature refinement (AFR) module. Specifically, AFR is designed as a two-branched network for simultaneous rain-distribution-aware attention map learning and attention guided hierarchy-preserving feature refinement. Guided by task-specific attention, coarse features are progressively refined to better model the diversified rainy effects. By using a separable convolution as the basic component, our AFR module introduces little computation overhead and can be readily integrated into most rainy-to-clean image translation networks for achieving better deraining results. By incorporating a series of AFR modules into a general encoder-decoder network, AFR-Net is constructed for deraining and it achieves new state-of-the-art results on both synthetic and real images. Furthermore, by using AFR-Net as a teacher model, we explore the use of knowledge distillation to successfully learn a student model that is also able to achieve state-of-the-art results but with a much faster inference speed (i.e., it only takes 0.08 second to process a 512×512 rainy image). Code and pre-trained models are available at (https://github.com/RobinCSIRO/AFR-Net).Keywords
Funding Information
- University of New South Wales
- CSIRO Data61 scholarships
This publication has 31 references indexed in Scilit:
- Deep Residual Learning for Image Recognition2016
- Rain Streak Removal Using Layer Priors2016
- Removing Rain from a Single Image via Discriminative Sparse Coding2015
- Adherent Raindrop Modeling, Detection and Removal in VideoIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Utilizing Local Phase Information to Remove Rain from VideoInternational Journal of Computer Vision, 2014
- A Generalized Low-Rank Appearance Model for Spatio-temporally Correlated Rain Streaks2013
- Restoring an Image Taken through a Window Covered with Dirt or Rain2013
- Automatic Single-Image-Based Rain Streaks Removal via Image DecompositionIEEE Transactions on Image Processing, 2011
- Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of StreaksInternational Journal of Computer Vision, 2011
- Vision and RainInternational Journal of Computer Vision, 2007