Differentiable Pooling for Hierarchical Feature Learning
Preprint
Abstract
We introduce a parametric form of pooling, based on a Gaussian, which can be optimized alongside the features in a single global objective function. By contrast, existing pooling schemes are based on heuristics (e.g. local maximum) and have no clear link to the cost function of the model. Furthermore, the variables of the Gaussian explicitly store location information, distinct from the appearance captured by the features, thus providing a what/where decomposition of the input signal. Although the differentiable pooling scheme can be incorporated in a wide range of hierarchical models, we demonstrate it in the context of a Deconvolutional Network model (Zeiler et al. ICCV 2011). We also explore a number of secondary issues within this model and present detailed experiments on MNIST digits.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- The Shape Boltzmann Machine: A Strong Model of Object ShapeInternational Journal of Computer Vision, 2013
- Adaptive deconvolutional networks for mid and high level feature learning2011
- Are sparse representations really relevant for image classification?2011
- Latent hierarchical structural learning for object detection2010
- Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations2009
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse ProblemsSIAM Journal on Imaging Sciences, 2009
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006
- Modeling receptive fields with non-negative sparse codingNeurocomputing, 2003
- Sparse coding with an overcomplete basis set: A strategy employed by V1?Vision Research, 1997
- Regression Shrinkage and Selection Via the LassoJournal of the Royal Statistical Society Series B: Statistical Methodology, 1996