Detection of correlations
Authors:
- Ery Arias-CastroEAEry Arias-Castro
- Sébastien BubeckSBSébastien Bubeck
- Gábor LugosiGLGábor Lugosi
Open Access
- 1 February 2012
- journal article
- research article
- Published by Institute of Mathematical Statistics in The Annals of Statistics
- Vol. 40 (1) , 412-435
Abstract
We consider the hypothesis testing problem of deciding whether an observed high-dimensional vector has independent normal components or, alternatively, if it has a small subset of correlated components. The correlated components may have a certain combinatorial structure known to the statistician. We establish upper and lower bounds for the worst-case (minimax) risk in terms of the size of the correlated subset, the level of correlation, and the structure of the class of possibly correlated sets. We show that some simple tests have near-optimal performance in many cases, while the generalized likelihood ratio test is suboptimal in some important cases.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Detection of an anomalous cluster in a networkThe Annals of Statistics, 2011
- On combinatorial testing problemsThe Annals of Statistics, 2010
- Optimal rates of convergence for covariance matrix estimationThe Annals of Statistics, 2010
- Innovated higher criticism for detecting sparse signals in correlated noiseThe Annals of Statistics, 2010
- Covariance regularization by thresholdingThe Annals of Statistics, 2008
- Searching for a trail of evidence in a mazeThe Annals of Statistics, 2008
- On the asymptotic distribution of the discrete scan statisticJournal of Applied Probability, 2006
- False Discovery Control for Random FieldsJournal of the American Statistical Association, 2004
- Maximal meaningful events and applications to image analysisThe Annals of Statistics, 2003
- Detection of stochastic processesIEEE Transactions on Information Theory, 1998