Towards Ground Truth for Single Image Deraining

Authors:
  • YBYunhao Ba
    Yunhao Ba
  • HZHoward Zhang
    Howard Zhang
  • EYEthan Yang
    Ethan Yang
  • ASAkira Suzuki
    Akira Suzuki
  • APArnold Pfahnl
    Arnold Pfahnl
  • CCChethan Chinder Chandrappa
    Chethan Chinder Chandrappa
  • CMCelso de Melo
    Celso de Melo
  • SYSuya You
    Suya You
  • SSStefano Soatto
    Stefano Soatto
  • AWAlex Wong
    Alex Wong
  • AKAchuta Kadambi
    Achuta Kadambi

Abstract
We propose a large-scale dataset of real-world rainy and clean image pairs and a method to remove degradations, induced by rain streaks and rain accumulation, from the image. As there exists no real-world dataset for deraining, current state-of-the-art methods rely on synthetic data and thus are limited by the sim2real domain gap; moreover, rigorous evaluation remains a challenge due to the absence of a real paired dataset. We fill this gap by collecting the first real paired deraining dataset through meticulous control of non-rain variations. Our dataset enables paired training and quantitative evaluation for diverse real-world rain phenomena (e.g. rain streaks and rain accumulation). To learn a representation invariant to rain phenomena, we propose a deep neural network that reconstructs the underlying scene by minimizing a rain-invariant loss between rainy and clean images. Extensive experiments demonstrate that the proposed dataset benefits existing derainers, and our model can outperform the state-of-the-art deraining methods on real rainy images under various conditions.