A hybrid linear/nonlinear training algorithm for feedforward neural networks
Authors:
- S. McLooneSMS. McLoone
- M D BrownMBM D Brown
- G. IrwinGIG. Irwin
- A. LightbodyALA. Lightbody
- 1 July 1998
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 9 (4) , 669-684
Abstract
This paper presents a new hybrid optimization strategy for training feedforward neural networks. The algorithm combines gradient-based optimization of nonlinear weights with singular value decomposition (SVD) computation of linear weights in one integrated routine. It is described for the multilayer perceptron (MLP) and radial basis function (RBF) networks and then extended to the local model network (LMN), a new feedforward structure in which a global nonlinear model is constructed from a set of locally valid submodels. Simulation results are presented demonstrating the superiority of the new hybrid training scheme compared to second-order gradient methods. It is particularly effective for the LMN architecture where the linear to nonlinear parameter ratio is large.Keywords
This publication has 20 references indexed in Scilit:
- Fast parallel off-line training of multilayer perceptronsIEEE Transactions on Neural Networks, 1997
- Design of optimal neurocontrollers for the separately excited dc motor using a hybrid genetic algorithm-neural network approach1996
- Nonlinear predictive control using local models — applied to a batch fermentation processControl Engineering Practice, 1995
- Identification of non-linear system structure and parameters using regime decompositionAutomatica, 1995
- Fast Gradient Based Off-Line Training of Multilayer Perceptrons1995
- Constructing NARMAX models using ARMAX modelsInternational Journal of Control, 1993
- Hybrid learning algorithm for Gaussian potential function networksIEE Proceedings D Control Theory and Applications, 1993
- Avoiding false local minima by proper initialization of connectionsIEEE Transactions on Neural Networks, 1992
- Artificial neural networks in process engineeringIEE Proceedings D Control Theory and Applications, 1991
- Increased rates of convergence through learning rate adaptationNeural Networks, 1988