Feature selection using linear classifier weights
Authors:
Open Access
- 25 July 2004
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 234-241
Abstract
This paper explores feature scoring and selection based on weights from linear classification models. It investigates how these methods combine with various learning models. Our comparative analysis includes three learning algorithms: Naïve Bayes, Perceptron, and Support Vector Machines (SVM) in combination with three feature weighting methods: Odds Ratio, Information Gain, and weights from linear models, the linear SVM and Perceptron. Experiments show that feature selection using weights from linear SVMs yields better classification performance than other feature weighting methods when combined with the three explored learning algorithms. The results support the conjecture that it is the sophistication of the feature weighting method rather than its apparent compatibility with the learning algorithm that improves classification performance.Keywords
This publication has 4 references indexed in Scilit:
- Information Theoretic Feature Crediting in Multiclass Support Vector Machines2001
- Support-Vector NetworksMachine Learning, 1995
- Constructing Decision Trees1993
- Learning algorithms with optimal stability in neural networksJournal of Physics A: General Physics, 1987