Random-projection Ensemble Classification
Open Access
- 30 June 2017
- journal article
- Published by Oxford University Press (OUP) in Journal of the Royal Statistical Society Series B: Statistical Methodology
- Vol. 79 (4) , 959-1035
Abstract
Summary: We introduce a very general method for high dimensional classification, based on careful combination of the results of applying an arbitrary base classifier to random projections of the feature vectors into a lower dimensional space. In one special case that we study in detail, the random projections are divided into disjoint groups, and within each group we select the projection yielding the smallest estimate of the test error. Our random-projection ensemble classifier then aggregates the results of applying the base classifier on the selected projections, with a data-driven voting threshold to determine the final assignment. Our theoretical results elucidate the effect on performance of increasing the number of projections. Moreover, under a boundary condition that is implied by the sufficient dimension reduction assumption, we show that the test excess risk of the random-projection ensemble classifier can be controlled by terms that do not depend on the original data dimension and a term that becomes negligible as the number of projections increases. The classifier is also compared empirically with several other popular high dimensional classifiers via an extensive simulation study, which reveals its excellent finite sample performance.Keywords
All Related Versions
Funding Information
- Engineering and Physical Sciences Research Council (EP/J017213/1)
- Philip Leverhulme Prize
This publication has 113 references indexed in Scilit:
- Penalized Classification using Fisher’s Linear DiscriminantJournal of the Royal Statistical Society Series B: Statistical Methodology, 2011
- Stability SelectionJournal of the Royal Statistical Society Series B: Statistical Methodology, 2010
- High-dimensional classification using features annealed independence rulesThe Annals of Statistics, 2008
- Sure Independence Screening for Ultrahigh Dimensional Feature SpaceJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- Model-Based Clustering, Discriminant Analysis, and Density EstimationJournal of the American Statistical Association, 2002
- Identifying Regression Outliers and Mixtures GraphicallyJournal of the American Statistical Association, 2000
- The random subspace method for constructing decision forestsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998
- Regularized Discriminant AnalysisJournal of the American Statistical Association, 1989
- Projection Pursuit RegressionJournal of the American Statistical Association, 1981
- The Efficiency of Logistic Regression Compared to Normal Discriminant AnalysisJournal of the American Statistical Association, 1975