Sparse Hebbian Learning is efficient with egalitarian homeostasis
Preprint
Abstract
Following the work of \citet{Olshausen98}, a number of unsupervised learning algorithms were proposed for receptive field formation in the primary visual cortex (V1). We describe here the theoretical formulation of a novel and simple algorithm aiming at achieving the shortest representation in an over-complete dictionary of unknown features using the L$_0$ norm, a "hard" NP-complete problem. Inspired by parallel computing solutions with spiking events, we propose here a solution based on correlation-based inhibition with a tuned egalitarian homeostasis. We present results of the simulation of this dynamical neural network with natural images and compare it to the {\sc SparseNet} solution. This algorithm exhibited similarly the formation of edge-like components as is observed in the input layer of V1 and to assess the quality of the different sets of filters, we additionally compared their relative efficiencies and show that our sub-optimal solution to the hard problem performs better than the optimal solution to the relaxed problem. We also show that homeostasis, by tuning the competition and cooperation, may yield solutions of different qualities which coding efficiency drastically depended on the goal assigned to the model by the cost function. This model provides a bridge between the non-linearity of the neural response and optimal use of distributed probabilistic representations of information. It is suggesting the importance of the role of interactions within neural assemblies \citep{Hebb49} to efficiently build representations in the neural code.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Optimality Model of Unsupervised Spike-Timing-Dependent Plasticity: Synaptic Memory and Weight DistributionNeural Computation, 2007
- A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fieldsJournal of Computational Neuroscience, 2006
- Efficient auditory codingNature, 2006
- Neurons Tune to the Earliest Spikes Through STDPNeural Computation, 2005
- Feature detection using spikes: The greedy approachJournal of Physiology-Paris, 2004
- Synaptic plasticity: taming the beastNature Neuroscience, 2000
- Learning the parts of objects by non-negative matrix factorizationNature, 1999
- What Is the Goal of Sensory Coding?Neural Computation, 1994
- The Role of Horizontal Connections in Generating Long Receptive Fields in the Cat Visual CortexEuropean Journal of Neuroscience, 1989
- A new look at the statistical model identificationIEEE Transactions on Automatic Control, 1974