A Joint Optimization of Momentum Item and Levenberg‐Marquardt Algorithm to Level Up the BPNN’s Generalization Ability
Authors:
Open Access
- 1 January 2014
- journal article
- research article
- Published by Wiley in Mathematical Problems in Engineering
- Vol. 2014 (1)
Abstract
Back propagation neural network (BPNN) as a kind of artificial neural network is widely used in pattern recognition and trend prediction. For standard BPNN, it has many drawbacks such as trapping into local optima, oscillation, and long training time. Because training the standard BPNN is based on gradient descent method, and the learning rate is fixed. Momentum item and Levenberg‐Marquardt (LM) algorithm are two ways to adjust the weights among the neurons and improve the BPNN’s performance. However, there is still much space to improve the two algorithms. The hybrid optimization of damping factor of LM and the dynamic momentum item is proposed in this paper. The improved BPNN is validated by Fisher Iris data and wine data. Then, it is used to predict the visit_spend. The database is provided by Dunnhumby′s Shopper Challenge. Compared with the other two improved BPNNs, the proposed method gets a better performance. Therefore, the proposed method can be used to do the pattern recognition and time series prediction more effectively.Keywords
Funding Information
- National Natural Science Foundation of China (51035008, 2009BB3365)
- Natural Science Foundation Project of Chongqing (51035008, 2009BB3365)
This publication has 19 references indexed in Scilit:
- Robust adaptive learning of feedforward neural networks via LMI optimizationsNeural Networks, 2012
- An Optimal PID Control Algorithm for Training Feedforward Neural NetworksIEEE Transactions on Industrial Electronics, 2012
- Application of Artificial Neural Networks in Prediction of Compressive Strength of Concrete by Using Ultrasonic Pulse VelocitiesIOSR Journal of Mechanical and Civil Engineering, 2012
- Convergence of Cyclic and Almost-Cyclic Learning With Momentum for Feedforward Neural NetworksIEEE Transactions on Neural Networks, 2011
- A new convex objective function for the supervised learning of single-layer neural networksPattern Recognition, 2010
- A recurrent neural network classifier for Doppler ultrasound blood flow signalsPattern Recognition Letters, 2006
- A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line SearchSIAM Journal on Optimization, 2005
- Training the random neural network using quasi-Newton methodsEuropean Journal of Operational Research, 2000
- Analysis of the back-propagation algorithm with momentumIEEE Transactions on Neural Networks, 1994
- An Algorithm for Least-Squares Estimation of Nonlinear ParametersJournal of the Society for Industrial and Applied Mathematics, 1963