Covariance matrix testing in high dimension using random projections
Preprint
Abstract
Estimation and hypothesis tests for the covariance matrix in high dimensions is a challenging problem as the traditional multivariate asymptotic theory is no longer valid. When the dimension is larger than or increasing with the sample size, standard likelihood based tests for the covariance matrix have poor performance. Existing high dimensional tests are either computationally expensive or have very weak control of type I error. In this paper, we propose a test procedure, CRAMP, for testing hypotheses involving one or more covariance matrices using random projections. Projecting the high dimensional data randomly into lower dimensional subspaces alleviates of the curse of dimensionality, allowing for the use of traditional multivariate tests. An extensive simulation study is performed to compare CRAMP against asymptotics-based high dimensional test procedures. An application of the proposed method to two gene expression data sets is presented.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Projected tests for high-dimensional covariance matricesJournal of Statistical Planning and Inference, 2020
- Hypothesis testing for the identity of high-dimensional covariance matricesStatistics & Probability Letters, 2020
- High-dimensional statistical inference: Theoretical development to data analytics2020
- Tests for covariance matrices in high dimension with less sample sizeJournal of Multivariate Analysis, 2014
- Two-Sample Covariance Matrix Testing and Support Recovery in High-Dimensional and Sparse SettingsJournal of the American Statistical Association, 2013
- On testing for an identity covariance matrix when the dimensionality equals or exceeds the sample sizeJournal of Statistical Planning and Inference, 2012
- A new test for sphericity of the covariance matrix for high dimensional dataJournal of Multivariate Analysis, 2010
- A test for the equality of covariance matrices when the dimension is large relative to the sample sizesComputational Statistics & Data Analysis, 2007
- Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arraysProceedings of the National Academy of Sciences, 1999
- Extensions of Lipschitz mappings into a Hilbert space1984