An Optimal Constrained Pruning Strategy for Decision Trees
- 1 February 2009
- journal article
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in INFORMS Journal on Computing
- Vol. 21 (1) , 49-61
Abstract
This paper is concerned with the optimal constrained pruning of decision trees. We present a novel 0–1 programming model for pruning the tree to minimize some general penalty function based on the resulting leaf nodes, and show that this model possesses a totally unimodular structure that enables it to be solved as a shortest-path problem on an acyclic graph. Moreover, we prove that this problem can be solved in strongly polynomial time while incorporating an additional constraint on the number of residual leaf nodes. Furthermore, the framework of the proposed modeling approach renders it suitable to accommodate different (multiple) objective functions and side-constraints, and we identify various such modeling options that can be applied in practice. The developed methodology is illustrated using a numerical example to provide insights, and some computational results are presented to demonstrate the efficacy of solving generically constrained problems of this type. We also apply this technique to a large-scale transportation analysis and simulation system (TRANSIMS), and present related computational results using real data to exhibit the flexibility and effectiveness of the proposed approach.Keywords
This publication has 15 references indexed in Scilit:
- Diversification for better classification treesComputers & Operations Research, 2006
- Genetically Engineered Decision Trees: Population Diversity Produces Smarter TreesOperations Research, 2003
- A Genetic Algorithm-Based Approach for Building Accurate Decision TreesINFORMS Journal on Computing, 2003
- Simplifying decision trees: A surveyThe Knowledge Engineering Review, 1997
- Optimal pruning for tree-structured vector quantizationInformation Processing & Management, 1992
- On the equivalence between some shortest path algorithmsOperations Research Letters, 1991
- A survey of decision tree classifier methodologyIEEE Transactions on Systems, Man, and Cybernetics, 1991
- Inferring decision trees using the minimum description lenght principleInformation and Computation, 1989
- Optimal pruning with applications to tree-structured source coding and modelingIEEE Transactions on Information Theory, 1989
- Preemptive and nonpreemptive multi-objective programming: Relationship and counterexamplesJournal of Optimization Theory and Applications, 1983