Simultaneous training of negatively correlated neural networks in an ensemble
Authors:
- 1 January 1999
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
- Vol. 29 (6) , 716-725
Abstract
This paper presents a new cooperative ensemble learning system (CELS) for designing neural network ensembles. The idea behind CELS is to encourage different individual networks in an ensemble to learn different parts or aspects of a training data so that the ensemble can learn the whole training data better. In CELS, the individual networks are trained simultaneously rather than independently or sequentially. This provides an opportunity for the individual networks to interact with each other and to specialize. CELS can create negatively correlated neural networks using a correlation penalty term in the error function to encourage such specialization. This paper analyzes CELS in terms of bias-variance-covariance tradeoff. CELS has also been tested on the Mackey-Glass time series prediction problem and the Australian credit card assessment problem. The experimental results show that CELS can produce neural network ensembles with good generalization ability.Keywords
This publication has 16 references indexed in Scilit:
- Making use of population information in evolutionary artificial neural networksIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1998
- A new evolutionary system for evolving artificial neural networksIEEE Transactions on Neural Networks, 1997
- Ensemble Learning Using Decorrelated Neural NetworksConnection Science, 1996
- Boosting and Other Ensemble MethodsNeural Computation, 1994
- 'Neural-gas' network for vector quantization and its application to time-series predictionIEEE Transactions on Neural Networks, 1993
- Neural Networks and the Bias/Variance DilemmaNeural Computation, 1992
- Fast Learning in Networks of Locally-Tuned Processing UnitsNeural Computation, 1989
- Predicting chaotic time seriesPhysical Review Letters, 1987
- Limits for the Precision and Value of Information from Dependent SourcesOperations Research, 1985
- Oscillation and Chaos in Physiological Control SystemsScience, 1977