Video Rain Streak Removal by Multiscale Convolutional Sparse Coding
Authors:
- Minghan LiMLMinghan Li
- Qi XieQXQi Xie
- Qian ZhaoQZQian Zhao
- Wei WeiWWWei Wei
- Shuhang GuSGShuhang Gu
- Jing TaoJTJing Tao
- Deyu MengDMDeyu Meng
- 1 June 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6644-6653
Abstract
Videos captured by outdoor surveillance equipments sometimes contain unexpected rain streaks, which brings difficulty in subsequent video processing tasks. Rain streak removal from a video is thus an important topic in recent computer vision research. In this paper, we raise two intrinsic characteristics specifically possessed by rain streaks. Firstly, the rain streaks in a video contain repetitive local patterns sparsely scattered over different positions of the video. Secondly, the rain streaks are with multiscale configurations due to their occurrence on positions with different distances to the cameras. Based on such understanding, we specifically formulate both characteristics into a multiscale convolutional sparse coding (MS-CSC) model for the video rain streak removal task. Specifically, we use multiple convolutional filters convolved on the sparse feature maps to deliver the former characteristic, and further use multiscale filters to represent different scales of rain streaks. Such a new encoding manner makes the proposed method capable of properly extracting rain streaks from videos, thus getting fine video deraining effects. Experiments implemented on synthetic and real videos verify the superiority of the proposed method, as compared with the state-of-the-art ones along this research line, both visually and quantitatively.Keywords
All Related Versions
This publication has 37 references indexed in Scilit:
- Utilizing Local Phase Information to Remove Rain from VideoInternational Journal of Computer Vision, 2014
- Improve robustness of sparse PCA by L1-norm maximizationPattern Recognition, 2012
- A Probabilistic Approach for Detection and Removal of Rain from VideosIETE Journal of Research, 2011
- Analysis of Rain and Snow in Frequency SpaceInternational Journal of Computer Vision, 2009
- Pixel Based Temporal Analysis Using Chromatic Property for Removing Rain from VideosComputer and Information Science, 2009
- Vision and RainInternational Journal of Computer Vision, 2007
- What energy functions can be minimized via graph cuts?IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Fast approximate energy minimization via graph cutsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- A model of saliency-based visual attention for rapid scene analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998