Linearized Multi-Sampling for Differentiable Image Transformation
Authors:
Preprint
Abstract
We propose a novel image sampling method for differentiable image transformation in deep neural networks. The sampling schemes currently used in deep learning, such as Spatial Transformer Networks, rely on bilinear interpolation, which performs poorly under severe scale changes, and more importantly, results in poor gradient propagation. This is due to their strict reliance on direct neighbors. Instead, we propose to generate random auxiliary samples in the vicinity of each pixel in the sampled image, and create a linear approximation with their intensity values. We then use this approximation as a differentiable formula for the transformed image. We demonstrate that our approach produces more representative gradients with a wider basin of convergence for image alignment, which leads to considerable performance improvements when training networks for classification tasks. This is not only true under large downsampling, but also when there are no scale changes. We compare our approach with multi-scale sampling and show that we outperform it. We then demonstrate that our improvements to the sampler are compatible with other tangential improvements to Spatial Transformer Networks and that it further improves their performance.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- Repeatability Is Not Enough: Learning Affine Regions via Discriminability2018
- LIFT: Learned Invariant Feature Transform2016
- Recurrent Attentional Networks for Saliency Detection2016
- Learning to Assign Orientations to Feature Points2016
- Fits Like a Glove: Rapid and Reliable Hand Shape Personalization2016
- Learning to Match Aerial Images with Deep Attentive Architectures2016
- DenseCap: Fully Convolutional Localization Networks for Dense Captioning2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Lucas-Kanade 20 Years On: A Unifying FrameworkInternational Journal of Computer Vision, 2004
- Performance of optical flow techniques2003