Hypothesis Testing in High-Dimensional Regression under the Gaussian Random Design Model: Asymptotic Theory
Authors:
Preprint
Abstract
We consider linear regression in the high-dimensional regime in which the number of observations $n$ is smaller than the number of parameters $p$. A very successful approach in this setting uses $\ell_1$-penalized least squares (a.k.a. the Lasso) to search for a subset of $s_0< n$ parameters that best explain the data, while setting the other parameters to zero. A considerable amount of work has been devoted to characterizing the estimation and model selection problems within this approach. In this paper we consider instead the fundamental, but far less understood, question of statistical significance. We study this problem under the random design model in which the rows of the design matrix are i.i.d. and drawn from a high-dimensional Gaussian distribution. This situation arises, for instance, in learning high-dimensional Gaussian graphical models. Leveraging on an asymptotic distributional characterization of regularized least squares estimators, we develop a procedure for computing p-values and hence assessing statistical significance for hypothesis testing. We characterize the statistical power of this procedure, and evaluate it on synthetic and real data, comparing it with earlier proposals. Finally, we provide an upper bound on the minimax power of tests with a given significance level and show that our proposed procedure achieves this bound in case of design matrices with i.i.d. Gaussian entries.
Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- State evolution for general approximate message passing algorithms, with applications to spatial couplingInformation and Inference: A Journal of the IMA, 2013
- Typicall1-recovery limit of sparse vectors represented by concatenations of random orthogonal matricesJournal of Statistical Mechanics: Theory and Experiment, 2012
- Generalized approximate message passing for estimation with random linear mixing2011
- Statistical mechanical analysis of compressed sensing utilizing correlated compression matrix2010
- Message-passing algorithms for compressed sensingProceedings of the National Academy of Sciences, 2009
- High-Dimensional Centrally Symmetric Polytopes with Neighborliness Proportional to DimensionDiscrete & Computational Geometry, 2005
- Neighborliness of randomly projected simplices in high dimensionsProceedings of the National Academy of Sciences, 2005
- Randomly Spread CDMA: Asymptotics Via Statistical PhysicsIEEE Transactions on Information Theory, 2005
- A statistical-mechanics approach to large-system analysis of CDMA multiuser detectorsIEEE Transactions on Information Theory, 2002
- Spin Glass Theory and BeyondPhysics Today, 1988