A New Correntropy-Based Conjugate Gradient Backpropagation Algorithm for Improving Training in Neural Networks
- 10 May 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 29 (12) , 6252-6263
Abstract
Mean square error (MSE) is the most prominent criterion in training neural networks and has been employed in numerous learning problems. In this paper, we suggest a group of novel robust information theoretic backpropagation (BP) methods, as correntropy-based conjugate gradient BP (CCG-BP). CCG-BP algorithms converge faster than the common correntropy-based BP algorithms and have better performance than the common CG-BP algorithms based on MSE, especially in nonGaussian environments and in cases with impulsive noise or heavy-tailed distributions noise. In addition, a convergence analysis of this new type of method is particularly considered. Numerical results for several samples of function approximation, synthetic function estimation, and chaotic time series prediction illustrate that our new BP method is more robust than the MSE-based method in the sense of impulsive noise, especially when SNR is low.Keywords
Funding Information
- Ferdowsi University of Mashhad
This publication has 44 references indexed in Scilit:
- On the Smoothed Minimum Error Entropy CriterionEntropy, 2012
- Correntropy based feature selection using binary projectionPattern Recognition, 2011
- Bias reduction in kernel density estimation via Lipschitz conditionJournal of Nonparametric Statistics, 2010
- New training strategies for constructive neural networks with application to regression problemsNeural Networks, 2004
- Confidence estimation methods for neural networks: a practical comparisonIEEE Transactions on Neural Networks, 2001
- Training feedforward networks with the Marquardt algorithmIEEE Transactions on Neural Networks, 1994
- Optimization for training neural netsIEEE Transactions on Neural Networks, 1992
- Conjugate gradient algorithm for efficient training of artificial neural networksIEE Proceedings G Circuits, Devices and Systems, 1992
- Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line SearchIMA Journal of Numerical Analysis, 1985
- Function minimization by conjugate gradientsThe Computer Journal, 1964