Generalized Linear Discriminant Analysis: A Unified Framework and Efficient Model Selection
Authors:
- 26 September 2008
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 19 (10) , 1768-1782
Abstract
High-dimensional data are common in many domains, and dimensionality reduction is the key to cope with the curse-of-dimensionality. Linear discriminant analysis (LDA) is a well-known method for supervised dimensionality reduction. When dealing with high-dimensional and low sample size data, classical LDA suffers from the singularity problem. Over the years, many algorithms have been developed to overcome this problem, and they have been applied successfully in various applications. However, there is a lack of a systematic study of the commonalities and differences of these algorithms, as well as their intrinsic relationships. In this paper, a unified framework for generalized LDA is proposed, which elucidates the properties of various algorithms and their relationships. Based on the proposed framework, we show that the matrix computations involved in LDA-based algorithms can be simplified so that the cross-validation procedure for model selection can be performed efficiently. We conduct extensive experiments using a collection of high-dimensional data sets, including text documents, face images, gene expression data, and gene expression pattern images, to evaluate the proposed theories and algorithms.Keywords
This publication has 25 references indexed in Scilit:
- Regularized linear discriminant analysis and its application in microarraysBiostatistics, 2006
- A unified framework for subspace face recognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2004
- An optimization criterion for generalized discriminant analysis on undersampled problemsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2004
- Prediction of central nervous system embryonal tumour outcome based on gene expressionNature, 2002
- PCA versus LDAIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- Nonlinear Dimensionality Reduction by Locally Linear EmbeddingScience, 2000
- A Global Geometric Framework for Nonlinear Dimensionality ReductionScience, 2000
- A new LDA-based face recognition system which can solve the small sample size problemPattern Recognition, 2000
- Eigenfaces vs. Fisherfaces: recognition using class specific linear projectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 1997
- An optimal transformation for discriminant and principal component analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 1988