Ridge Regression Under Dense Factor Augmented Models
Author:
Open Access
- 5 June 2023
- journal article
- research article
- Published by Taylor & Francis in Journal of the American Statistical Association
- Vol. 119 (546) , 1566-1578
Abstract
This paper establishes a comprehensive theory of the optimality, robustness, and cross-validation selection consistency for the ridge regression under factor-augmented models with possibly dense idiosyncratic information. Using spectral analysis for random matrices, we show that the ridge regression is asymptotically efficient in capturing both factor and idiosyncratic information by minimizing the limiting predictive loss among the entire class of spectral regularized estimators under large-dimensional factor models and mixed-effects hypothesis. We derive an asymptotically optimal ridge penalty in closed form and prove that a bias-corrected k-fold cross-validation procedure can adaptively select the best ridge penalty in large samples. We extend the theory to the autoregressive models with many exogenous variables and establish a consistent cross-validation procedure using the what-we-called double ridge regression method. Our results allow for non-parametric distributions for, possibly heavy-tailed, martingale difference errors and idiosyncratic random coefficients and adapt to the cross-sectional and temporal dependence structures of the large-dimensional predictors. We demonstrate the performance of our ridge estimators in simulated examples as well as an economic dataset. All the proofs are available in the supplement, which also includes more technical discussions and remarks, extra simulation results, and useful lemmas that may be of independent interest.Keywords
This publication has 43 references indexed in Scilit:
- Asymptotics of the principal components estimator of large factor models with weakly influential factorsJournal of Econometrics, 2012
- Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control ProgramJournal of the American Statistical Association, 2010
- PCA consistency in high dimension, low sample size contextThe Annals of Statistics, 2009
- On Consistency and Sparsity for Principal Components Analysis in High DimensionsJournal of the American Statistical Association, 2009
- Forecasting using a large number of predictors: Is Bayesian shrinkage a valid alternative to principal components?Journal of Econometrics, 2008
- Basic Properties of Strong Mixing Conditions. A Survey and Some Open QuestionsProbability Surveys, 2005
- Active Shape Models-Their Training and ApplicationComputer Vision and Image Understanding, 1995
- Bayesian Variable Selection in Linear RegressionJournal of the American Statistical Association, 1988
- The Law of Large Numbers for Subsequences of a Stationary ProcessThe Annals of Probability, 1975
- Ridge Regression: Biased Estimation for Nonorthogonal ProblemsTechnometrics, 1970