A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data
- 1 July 2015
- journal article
- Published by MIT Press in Neural Computation
- Vol. 27 (7) , 1461-1495
Abstract
Neural network models of early sensory processing typically reduce the dimensionality of streaming input data. Such networks learn the principal subspace, in the sense of principal component analysis, by adjusting synaptic weights according to activity-dependent learning rules. When derived from a principled cost function, these rules are nonlocal and hence biologically implausible. At the same time, biologically plausible local rules have been postulated rather than derived from a principled cost function. Here, to bridge this gap, we derive a biologically plausible network for subspace learning on streaming data by minimizing a principled cost function. In a departure from previous work, where cost was quantified by the representation, or reconstruction, error, we adopt a multidimensional scaling cost function for streaming data. The resulting algorithm relies only on biologically plausible Hebbian and anti-Hebbian local learning rules. In a stochastic setting, synaptic weights converge to a stationary state, which projects the input data onto the principal subspace. If the data are generated by a nonstationary distribution, the network can track the principal subspace. Thus, our result makes a step toward an algorithmic theory of neural computation.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- On the discrete-time dynamics of the basic Hebbian neural network nodeIEEE Transactions on Neural Networks, 2002
- Projection approximation subspace trackingIEEE Transactions on Signal Processing, 1995
- Lyapunov functions for convergence of principal component algorithmsNeural Networks, 1995
- Adaptive Principal component EXtraction (APEX) and applicationsIEEE Transactions on Signal Processing, 1994
- Principal components, minor components, and linear neural networksNeural Networks, 1992
- Dynamics of learning in linear feature-discovery networksNetwork: Computation in Neural Systems, 1991
- A Self-Organizing Network for Principal-Component AnalysisEurophysics Letters, 1989
- Simplified neuron model as a principal component analyzerJournal of Mathematical Biology, 1982
- On the Convergence of the Classical Iterative Method of Solving Linear Simultaneous EquationsThe Annals of Mathematical Statistics, 1949
- LIII. On lines and planes of closest fit to systems of points in spaceJournal of Computers in Education, 1901