Performing Feature Selection With Multilayer Perceptrons
Authors:
- Enrique RomeroEREnrique Romero
- Josep María SopenaJSJosep María Sopena
- 26 February 2008
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 19 (3) , 431-441
Abstract
An experimental study on two decision issues for wrapper feature selection (FS) with multilayer perceptrons and the sequential backward selection (SBS) procedure is presented. The decision issues studied are the stopping criterion and the network retraining before computing the saliency. Experimental results indicate that the increase in the computational cost associated with retraining the network with every feature temporarily removed before computing the saliency is rewarded with a significant performance improvement. Despite being quite intuitive, this idea has been hardly used in practice. A somehow nonintuitive conclusion can be drawn by looking at the stopping criterion, suggesting that forcing overtraining may be as useful as early stopping. A significant improvement in the overall results with respect to learning with the whole set of variables is observed.Keywords
This publication has 36 references indexed in Scilit:
- Input feature selection for classification problemsIEEE Transactions on Neural Networks, 2002
- A new pruning heuristic based on variance analysis of sensitivity informationIEEE Transactions on Neural Networks, 2001
- Anisotropic noise injection for input variables relevance determinationIEEE Transactions on Neural Networks, 2000
- Perturbation method for deleting redundant inputs of perceptron networksNeurocomputing, 1997
- Variable selection with neural networksNeurocomputing, 1996
- Floating search methods in feature selectionPattern Recognition Letters, 1994
- Bayesian selection of important features for feedforward neural networksNeurocomputing, 1993
- Pruning algorithms-a surveyIEEE Transactions on Neural Networks, 1993
- Neural Networks and the Bias/Variance DilemmaNeural Computation, 1992
- Orthogonal least squares methods and their application to non-linear system identificationInternational Journal of Control, 1989