Robust Spectral Learning for Unsupervised Feature Selection
Authors:
- Lei ShiLSLei Shi
- Liang DuLDLiang Du
- Yi-Dong ShenYSYi-Dong Shen
- 1 December 2014
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15504786,p. 977-982
Abstract
In this paper, we consider the problem of unsupervised feature selection. Recently, spectral feature selection algorithms, which leverage both graph Laplacian and spectral regression, have received increasing attention. However, existing spectral feature selection algorithms suffer from two major problems: 1) since the graph Laplacian is constructed from the original feature space, noisy and irrelevant features may have adverse effect on the estimated graph Laplacian and hence degenerate the quality of the induced graph embedding, 2) since the cluster labels are discrete in natural, relaxing and approximating these labels into a continuous embedding can inevitably introduce noise into the estimated cluster labels. Without considering the noise in the cluster labels, the feature selection process may be misguided. In this paper, we propose a Robust Spectral learning framework for unsupervised Feature Selection (RSFS), which jointly improves the robustness of graph embedding and sparse spectral regression. Compared with existing methods which are sensitive to noisy features, our proposed method utilizes a robust local learning method to construct the graph Laplacian and a robust spectral regression method to handle the noise on the learned cluster labels. In order to solve the proposed optimization problem, an efficient iterative algorithm is proposed. We also show the close connection between the proposed robust spectral regression and robust Huber M-estimator. Experimental results on different datasets show the superiority of RSFS.Keywords
This publication has 13 references indexed in Scilit:
- Local and Global Discriminative Learning for Unsupervised Feature Selection2013
- Optimization with Sparsity-Inducing PenaltiesFoundations and Trends in Machine Learning, 2012
- Robust principal component analysis?Journal of the ACM, 2011
- Co-clustering on manifolds2009
- Clustering Via Local Regression2008
- Spectral feature selection for supervised and unsupervised learning2007
- Producing Accurate Interpretable Clusters from High-Dimensional Data2005
- Normalized cuts and image segmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
- Automatic classification of single facial imagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 1999
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998