Rethinking Multi-Scale Representations in Deep Deraining Transformer
Open Access
- 24 March 2024
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (2) , 1046-1053
Abstract
Existing Transformer-based image deraining methods depend mostly on fixed single-input single-output U-Net architecture. In fact, this not only neglects the potentially explicit information from multiple image scales, but also lacks the capability of exploring the complementary implicit information across different scales. In this work, we rethink the multi-scale representations and design an effective multi-input multi-output framework that constructs intra- and inter-scale hierarchical modulation to better facilitate rain removal and help image restoration. We observe that rain levels reduce dramatically in coarser image scales, thus proposing to restore rain-free results from the coarsest scale to the finest scale in image pyramid inputs, which also alleviates the difficulty of model learning. Specifically, we integrate a sparsity-compensated Transformer block and a frequency-enhanced convolutional block into a coupled representation module, in order to jointly learn the intra-scale content-aware features. To facilitate representations learned at different scales to communicate with each other, we leverage a gated fusion module to adaptively aggregate the inter-scale spatial-aware features, which are rich in correlated information of rain appearances, leading to high-quality results. Extensive experiments demonstrate that our model achieves consistent gains on five benchmarks.Keywords
This publication has 30 references indexed in Scilit:
- Spatial Attentive Single-Image Deraining With a High Quality Real Rain Dataset2019
- Progressive Image Deraining Networks: A Better and Simpler Baseline2019
- Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining2018
- Density-Aware Single Image De-raining Using a Multi-stream Dense Network2018
- Deep Joint Rain Detection and Removal from a Single Image2017
- Removing Rain from Single Images via a Deep Detail Network2017
- Rain Streak Removal Using Layer Priors2016
- Blind Deblurring Using Internal Patch Recurrence2014
- Separating Signal from Noise Using Patch Recurrence across Scales2013
- Scope of validity of PSNR in image/video quality assessmentElectronics Letters, 2008