Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Analysis and a New Strategy
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8372-8381
Abstract
Data augmentation is an effective way to improve the performance of deep networks. Unfortunately, current methods are mostly developed for high-level vision tasks (e.g., classification) and few are studied for low-level vision tasks (e.g., image restoration). In this paper, we provide a comprehensive analysis of the existing augmentation methods applied to the super-resolution task. We find that the methods discarding or manipulating the pixels or features too much hamper the image restoration, where the spatial relationship is very important. Based on our analyses, we propose CutBlur that cuts a low-resolution patch and pastes it to the corresponding high-resolution image region and vice versa. The key intuition of CutBlur is to enable a model to learn not only "how" but also "where" to super-resolve an image. By doing so, the model can understand "how much", instead of blindly learning to apply super-resolution to every given pixel. Our method consistently and significantly improves the performance across various scenarios, especially when the model size is big and the data is collected under real-world environments. We also show that our method improves other low-level vision tasks, such as denoising and compression artifact removal.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results2017
- NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study2017
- Sketch-based manga retrieval using manga109 datasetMultimedia Tools and Applications, 2016
- Rethinking the Inception Architecture for Computer Vision2016
- Seven Ways to Improve Example-Based Single Image Super Resolution2016
- Convolutional Sparse Coding for Image Super-Resolution2015
- Efficient object localization using Convolutional Networks2015
- Single image super-resolution from transformed self-exemplars2015
- Image Super-Resolution Using Deep Convolutional NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Image Super-Resolution Via Sparse RepresentationIEEE Transactions on Image Processing, 2010