A New BP Algorithm with Adaptive Momentum for FNNs Training
Authors:
- Hongmei ShaoHSHongmei Shao
- Gaofeng ZhengGZGaofeng Zheng
- 1 January 2009
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 4 (21556083) , 16-20
Abstract
In this paper, a new back propagation (BP) algorithm with adaptive momentum is proposed, where the momentum coefficient is adjusted iteratively based on the current descent direction and the weight increment in the last iteration. A convergence result of the algorithm is presented when it is used for training feed forward neural networks (FNNs) with a hidden layer. Simulation results have shown that this new algorithm has a distinct superiority in fast convergence and smoothing oscillation over the conventional BP method. Moreover, the range for the learning rate has been widened after the inclusion of such an adaptable momentum while maintaining the stability of networks.Keywords
This publication has 9 references indexed in Scilit:
- Variable-step BP training algorithm with an adaptive momentum term2003
- A new acceleration technique for the backpropagation algorithm2002
- The need for small learning rates on large problems2002
- IMPROVED BACKPROPAGATION LEARNING IN NEURAL NETWORKS WITH WINDOWED MOMENTUMInternational Journal of Neural Systems, 2002
- Simple adaptive momentum: New algorithm for trainingmultilayer perceptronsElectronics Letters, 1994
- Accelerated training of backpropagation networks by using adaptive momentum stepElectronics Letters, 1992
- An adaptive training algorithm for back propagation networksComputer Speech & Language, 1987
- Learning representations by back-propagating errorsNature, 1986
- Parallel Distributed Processing1986