Global and Local Structure Preservation for Feature Selection
- 7 November 2013
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 25 (6) , 1083-1095
Abstract
The recent literature indicates that preserving global pairwise sample similarity is of great importance for feature selection and that many existing selection criteria essentially work in this way. In this paper, we argue that besides global pairwise sample similarity, the local geometric structure of data is also critical and that these two factors play different roles in different learning scenarios. In order to show this, we propose a global and local structure preservation framework for feature selection (GLSPFS) which integrates both global pairwise sample similarity and local geometric data structure to conduct feature selection. To demonstrate the generality of our framework, we employ methods that are well known in the literature to model the local geometric data structure and develop three specific GLSPFS-based feature selection algorithms. Also, we develop an efficient optimization algorithm with proven global convergence to solve the resulting feature selection problem. A comprehensive experimental study is then conducted in order to compare our feature selection algorithms with many state-of-the-art ones in supervised, unsupervised, and semisupervised learning scenarios. The result indicates that: 1) our framework consistently achieves statistically significant improvement in selection performance when compared with the currently used algorithms; 2) in supervised and semisupervised learning scenarios, preserving global pairwise similarity is more important than preserving local geometric data structure; 3) in the unsupervised scenario, preserving local geometric data structure becomes clearly more important; and 4) the best feature selection performance is always obtained when the two factors are appropriately integrated. In summary, this paper not only validates the advantages of the proposed GLSPFS framework but also gains more insight into the information to be preserved in different feature selection tasks.Keywords
Funding Information
- National Natural Science Foundation of China
- Australian Research Council Linkage (LP0991757)
This publication has 30 references indexed in Scilit:
- Local Coordinates Alignment With Global Preservation for Dimensionality ReductionIEEE Transactions on Neural Networks and Learning Systems, 2012
- A Discrimination Analysis for Unsupervised Feature Selection via Optic Diffraction PrincipleIEEE Transactions on Neural Networks and Learning Systems, 2012
- Feature Selection Using Probabilistic Prediction of Support Vector RegressionIEEE Transactions on Neural Networks, 2011
- Feature Selection With Redundancy-Constrained Class SeparabilityIEEE Transactions on Neural Networks, 2010
- Spectral feature selection for supervised and unsupervised learning2007
- Face recognition using LaplacianfacesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2005
- Theoretical and Empirical Analysis of ReliefF and RReliefFMachine Learning, 2003
- Nonlinear Dimension Reduction via Local Tangent Space Alignment2003
- Nonlinear Dimensionality Reduction by Locally Linear EmbeddingScience, 2000
- Feature selection: evaluation, application, and small sample performanceIEEE Transactions on Pattern Analysis and Machine Intelligence, 1997