IMPROVING STABILITY OF DECISION TREES
Authors:
- 1 March 2002
- journal article
- Published by World Scientific Pub Co Pte Ltd in International Journal of Pattern Recognition and Artificial Intelligence
- Vol. 16 (02) , 145-159
Abstract
Decision-tree algorithms are known to be unstable: small variations in the training set can result in different trees and different predictions for the same validation examples. Both accuracy and stability can be improved by learning multiple models from bootstrap samples of training data, but the "meta-learner" approach makes the extracted knowledge hardly interpretable. In the following paper, we present the Info-Fuzzy Network (IFN), a novel information-theoretic method for building stable and comprehensible decision-tree models. The stability of the IFN algorithm is ensured by restricting the tree structure to using the same feature for all nodes of the same tree level and by the built-in statistical significance tests. The IFN method is shown empirically to produce more compact and stable models than the "meta-learner" techniques, while preserving a reasonable level of predictive accuracy.Keywords
This publication has 6 references indexed in Scilit:
- Matching hierarchical structures using association graphsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1999
- Knowledge discovery via multiple modelsIntelligent Data Analysis: An International Journal, 1998
- Bagging predictorsMachine Learning, 1996
- Connectivity and performance tradeoffs in the cascade correlation learning architectureIEEE Transactions on Neural Networks, 1994
- Learning structural descriptions of patterns: A new technique for conditional clustering and rule generationPattern Recognition, 1994
- A machine learning method for generation of a neural network architecture: a continuous ID3 algorithmIEEE Transactions on Neural Networks, 1992