Large-scale attribute selection using wrappers
- 1 March 2009
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 332-339
Abstract
Scheme-specific attribute selection with the wrapper and variants of forward selection is a popular attribute selection technique for classification that yields good results. However, it can run the risk of overfitting because of the extent of the search and the extensive use of internal cross-validation. Moreover, although wrapper evaluators tend to achieve superior accuracy compared to filters, they face a high computational cost. The problems of overfitting and high runtime occur in particular on high-dimensional datasets, like microarray data. We investigate Linear Forward Selection, a technique to reduce the number of attributes expansions in each forward selection step. Our experiments demonstrate that this approach is faster, finds smaller subsets and can even increase the accuracy compared to standard forward selection. We also investigate a variant that applies explicit subset size determination in forward selection to combat overfitting, where the search is forced to stop at a precomputed ldquooptimalrdquo subset size. We show that this technique reduces subset size while maintaining comparable accuracy.Keywords
This publication has 17 references indexed in Scilit:
- A Hybrid Feature Selection Approach for Microarray Gene Expression DataLecture Notes in Computer Science, 2006
- Benchmarking attribute selection techniques for discrete class data miningIEEE Transactions on Knowledge and Data Engineering, 2003
- Gene-expression profiles predict survival of patients with lung adenocarcinomaNature Medicine, 2002
- The Use of Molecular Profiling to Predict Survival after Chemotherapy for Diffuse Large-B-Cell LymphomaNew England Journal of Medicine, 2002
- Gene expression correlates of clinical prostate cancer behaviorCancer Cell, 2002
- Prediction of central nervous system embryonal tumour outcome based on gene expressionNature, 2002
- Learning to remove Internet advertisements1999
- Wrappers for feature subset selectionArtificial Intelligence, 1997
- Floating search methods in feature selectionPattern Recognition Letters, 1994
- Feature selection and feature extraction for text categorization1992