Role of Homeostasis in Learning Sparse Representations
Open Access
- 1 July 2010
- journal article
- research article
- Published by MIT Press in Neural Computation
- Vol. 22 (7) , 1812-1836
Abstract
Neurons in the input layer of primary visual cortex in primates develop edge-like receptive fields. One approach to understanding the emergence of this response is to state that neural activity has to efficiently represent sensory data with respect to the statistics of natural scenes. Furthermore, it is believed that such an efficient coding is achieved using a competition across neurons so as to generate a sparse representation, that is, where a relatively small number of neurons are simultaneously active. Indeed, different models of sparse coding, coupled with Hebbian learning and homeostasis, have been proposed that successfully match the observed emergent response. However, the specific role of homeostasis in learning such sparse representations is still largely unknown. By quantitatively assessing the efficiency of the neural representation during learning, we derive a cooperative homeostasis mechanism that optimally tunes the competition between neurons within the sparse coding algorithm. We apply this homeostasis while learning small patches taken from natural images and compare its efficiency with state-of-the-art algorithms. Results show that while different sparse coding algorithms give similar coding results, the homeostasis provides an optimal balance for the representation of natural images within the population of neurons. Competition in sparse coding is optimized when it is fair. By contributing to optimizing statistical competition across neurons, homeostasis is crucial in providing a more efficient solution to the emergence of independent components.Keywords
All Related Versions
This publication has 46 references indexed in Scilit:
- Establishing a Novel Modeling Tool: A Python-based Interface for a Neuromorphic Hardware SystemFrontiers in Neuroinformatics, 2009
- Order-Based Representation in Random Networks of Cortical NeuronsPLOS Computational Biology, 2008
- Adaptive regulation of sparseness by feedforward inhibitionNature Neuroscience, 2007
- Sparse Approximation of Images Inspired from the Functional Architecture of the Primary Visual AreasEURASIP Journal on Advances in Signal Processing, 2006
- Efficient auditory codingNature, 2006
- Sparse spike coding in an asynchronous feed-forward multi-layer neural network using matching pursuitNeurocomputing, 2004
- Sparse coding with an overcomplete basis set: A strategy employed by V1?Vision Research, 1997
- Origin of orientation tuning in the visual cortexCurrent Opinion in Neurobiology, 1992
- Simplified neuron model as a principal component analyzerJournal of Mathematical Biology, 1982
- A new look at the statistical model identificationIEEE Transactions on Automatic Control, 1974