Adaptive, global, extended Kalman filters for training feedforward neural networks
Authors:
- 1 November 1994
- journal article
- research article
- Published by Wiley in Journal of Chemometrics
- Vol. 8 (6) , 391-407
Abstract
Finding methods for the optimization of weights in feedforward neural networks has become an ongoing developmental process in connectionist research. The current focus on finding new methods for the optimization of weights is mostly the result of the slow and unreliable convergence properties of the gradient descent optimization used in the original back‐propagation algorithm. More accurate and computationally expensive second‐order gradient methods have displaced earlier first‐order gradient optimization of the network connection weights. The global, extended Kalman filter is among the most accurate and computationally expensive of these second‐order weight optimization methods. The iterative, second‐order nature of the filter results in a large number of calculations for each sweep of the training set. This can increase the training time dramatically when training is conducted with data sets that contain large numbers of training patterns. In this paper an adaptive variant of the global, extended Kalman filter that exhibits substantially improved convergence properties is presented and discussed. The adaptive mechanism permits more rapid convergence of network training by identifying data that contain redundant information and avoiding calculations based on this redundant information.Keywords
This publication has 18 references indexed in Scilit:
- Nonlinear multivariate mapping of chemical data using feed-forward neural networksAnalytical Chemistry, 1993
- Data processing using neural networksAnalytica Chimica Acta, 1993
- Progress in supervised neural networksIEEE Signal Processing Magazine, 1993
- Optimal minimal neural interpretation of spectraAnalytical Chemistry, 1992
- Nonlinear multivariate calibration using principal components regression and artificial neural networksAnalytical Chemistry, 1991
- Neural Networks: An OverviewAnalytical Chemistry, 1991
- Identifying functional groups in IR spectra using an artificial neural networkJournal of the Chemical Society, Perkin Transactions 2, 1991
- Approximation by superpositions of a sigmoidal functionMathematics of Control, Signals, and Systems, 1989
- Nonlinear calibration using projection pursuit regression: application to an array of ion-selective electrodesAnalytical Chemistry, 1988
- ACE: A non-linear regression modelChemometrics and Intelligent Laboratory Systems, 1988