Intensity-Aware Single-Image Deraining With Semantic and Color Regularization
- 8 October 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 30 (10577149) , 8497-8509
Abstract
Rain degrades image visual quality and disrupts object structures, obscuring their details and erasing their colors. Existing deraining methods are primarily based on modeling either visual appearances of rain or its physical characteristics ( e.g. , rain direction and density), and thus suffer from two common problems. First, due to the stochastic nature of rain, they tend to fail in recognizing rain streaks correctly, and wrongly remove image structures and details. Second, they fail to recover the image colors erased by heavy rain. In this paper, we address these two problems with the following three contributions. First, we propose a novel PHP block to aggregate comprehensive spatial and hierarchical information for removing rain streaks of different sizes. Second, we propose a novel network to first remove rain streaks, then recover objects structures/colors, and finally enhance details. Third, to train the network, we prepare a new dataset, and propose a novel loss function to introduce semantic and color regularization for deraining. Extensive experiments demonstrate the superiority of the proposed method over state-of-the-art deraining methods on both synthesized and real-world data, in terms of visual quality, quantitative accuracy, and running speed.Keywords
Funding Information
- National Natural Science Foundation of China (61632006, 61972067)
- Innovation Technology Funding of Dalian (2018J11CY010, 2020JJ26GX036)
- General Research Fund from RGC of Hong Kong SAR (11205620)
This publication has 49 references indexed in Scilit:
- Clearing the Skies: A Deep Network Architecture for Single-Image Rain RemovalIEEE Transactions on Image Processing, 2017
- The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes2016
- Rain Streak Removal Using Layer Priors2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Removing Rain from a Single Image via Discriminative Sparse Coding2015
- Utilizing Local Phase Information to Remove Rain from VideoInternational Journal of Computer Vision, 2014
- A Generalized Low-Rank Appearance Model for Spatio-temporally Correlated Rain Streaks2013
- Automatic Single-Image-Based Rain Streaks Removal via Image DecompositionIEEE Transactions on Image Processing, 2011
- Vision and RainInternational Journal of Computer Vision, 2007
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004