Exploring constructive cascade networks
Authors:
- T.D. GedeonTGT.D. Gedeon
- 1 January 1999
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 10 (6) , 1335-1350
Abstract
Constructive algorithms have proved to be powerful methods for training feedforward neural networks. An important property of these algorithms is generalization. A series of empirical studies were performed to examine the effect of regularization on generalization in constructive cascade algorithms. It was found that the combination of early stopping and regularization resulted in better generalization than the use of early stopping alone. A cubic penalty term that greatly penalizes large weights was shown to be beneficial for generalization in cascade networks. An adaptive method of setting the regularization magnitude in constructive algorithms was introduced and shown to produce generalization results similar to those obtained with a fixed, user-optimized regularization setting. This adaptive method also resulted in the construction of smaller networks for more complex problems. The acasper algorithm, which incorporates the insights obtained from the empirical studies, was shown to have good generalization and network construction properties. This algorithm was compared to the cascade correlation algorithm on the Proben 1 and additional regression data sets.Keywords
This publication has 25 references indexed in Scilit:
- Simulated annealing and weight decay in adaptive learning: the SARPROP algorithmIEEE Transactions on Neural Networks, 1998
- Constructive algorithms for structure learning in feedforward neural networks for regression problemsIEEE Transactions on Neural Networks, 1997
- Use of bias term in projection pursuit learning improves approximation and convergence propertiesIEEE Transactions on Neural Networks, 1996
- The cascade-correlation learning: a projection pursuit learning perspectiveIEEE Transactions on Neural Networks, 1996
- Ridge polynomial networksIEEE Transactions on Neural Networks, 1995
- Advanced supervised learning in multi-layer perceptrons — From backpropagation to adaptive learning algorithmsComputer Standards & Interfaces, 1994
- Regression modeling in back-propagation and projection pursuit learningIEEE Transactions on Neural Networks, 1994
- Dynamic node architecture learning: An information theoretic approachNeural Networks, 1994
- Pruning algorithms-a surveyIEEE Transactions on Neural Networks, 1993
- Dynamic Node Creation in Backpropagation NetworksConnection Science, 1989