Complexity Issues in Natural Gradient Descent Method for Training Multilayer Perceptrons
Authors:
- 1 November 1998
- journal article
- Published by MIT Press in Neural Computation
- Vol. 10 (8) , 2137-2157
Abstract
The natural gradient descent method is applied to train an n-m-1 multilayer perceptron. Based on an efficient scheme to represent the Fisher information matrix for an n-m-1 stochastic multilayer perceptron, a new algorithm is proposed to calculate the natural gradient without inverting the Fisher information matrix explicitly. When the input dimension n is much larger than the number of hidden neurons m, the time complexity of computing the natural gradient is O(n).Keywords
This publication has 3 references indexed in Scilit:
- Natural Gradient Works Efficiently in LearningNeural Computation, 1998
- Statistical inference: learning in artificial neural networksTrends in Cognitive Sciences, 1998
- On-line learning in soft committee machinesPhysical Review E, 1995