Debiasing the lasso: Optimal sample size for Gaussian designs
Authors:
- Adel JavanmardAJAdel Javanmard
- Andrea MontanariAMAndrea Montanari
- 1 December 2018
- journal article
- research article
- Published by Institute of Mathematical Statistics in The Annals of Statistics
- Vol. 46 (6A) , 2593-2622
Abstract
Performing statistical inference in high-dimensional models is challenging because of the lack of precise information on the distribution of high-dimensional regularized estimators. Here, we consider linear regression in the high-dimensional regime $p>>n$ and the Lasso estimator: we would like to perform inference on the parameter vector $\theta^{*}\in\mathbb{R}^{p}$. Important progress has been achieved in computing confidence intervals and $p$-values for single coordinates $\theta^{*}_{i}$, $i\in\{1,\dots,p\}$. A key role in these new inferential methods is played by a certain debiased estimator $\widehat{\theta}^{\mathrm{d}}$. Earlier work establishes that, under suitable assumptions on the design matrix, the coordinates of $\widehat{\theta}^{\mathrm{d}}$ are asymptotically Gaussian provided the true parameters vector $\theta^{*}$ is $s_{0}$-sparse with $s_{0}=o(\sqrt{n}/\log p)$. The condition $s_{0}=o(\sqrt{n}/\log p)$ is considerably stronger than the one for consistent estimation, namely $s_{0}=o(n/\log p)$. In this paper, we consider Gaussian designs with known or unknown population covariance. When the covariance is known, we prove that the debiased estimator is asymptotically Gaussian under the nearly optimal condition $s_{0}=o(n/(\log p)^{2})$. The same conclusion holds if the population covariance is unknown but can be estimated sufficiently well. For intermediate regimes, we describe the trade-off between sparsity in the coefficients $\theta^{*}$, and sparsity in the inverse covariance of the design. We further discuss several applications of our results beyond high-dimensional inference. In particular, we propose a thresholded Lasso estimator that is minimax optimal up to a factor $1+o_{n}(1)$ for i.i.d. Gaussian designs.
Keywords
All Related Versions
This publication has 49 references indexed in Scilit:
- Stability SelectionJournal of the Royal Statistical Society Series B: Statistical Methodology, 2010
- Nearly unbiased variable selection under minimax concave penaltyThe Annals of Statistics, 2010
- Message-passing algorithms for compressed sensingProceedings of the National Academy of Sciences, 2009
- Simultaneous analysis of Lasso and Dantzig selectorThe Annals of Statistics, 2009
- Sure Independence Screening for Ultrahigh Dimensional Feature SpaceJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- The sparsity and bias of the Lasso selection in high-dimensional linear regressionThe Annals of Statistics, 2008
- On the “degrees of freedom” of the lassoThe Annals of Statistics, 2007
- Stable signal recovery from incomplete and inaccurate measurementsCommunications on Pure and Applied Mathematics, 2006
- Uncertainty principles and ideal atomic decompositionIEEE Transactions on Information Theory, 2001
- A new look at the statistical model identificationIEEE Transactions on Automatic Control, 1974