Analysis of Testing-Based Forward Model Selection
Author:
Preprint
Abstract
This paper introduces and analyzes a procedure called Testing-based forward model selection (TBFMS) in linear regression problems. This procedure inductively selects covariates that add predictive power into a working statistical model before estimating a final regression. The criterion for deciding which covariate to include next and when to stop including covariates is derived from a profile of traditional statistical hypothesis tests. This paper proves probabilistic bounds, which depend on the quality of the tests, for prediction error and the number of selected covariates. As an example, the bounds are then specialized to a case with heteroskedastic data, with tests constructed with the help of Huber-Eicker-White standard errors. Under the assumed regularity conditions, these tests lead to estimation convergence rates matching other common high-dimensional estimators including Lasso.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- Inference in High-Dimensional Panel Models With an Application to Gun ControlJournal of Business & Economic Statistics, 2016
- Sequential Selection Procedures and False Discovery Rate ControlJournal of the Royal Statistical Society Series B: Statistical Methodology, 2015
- Inference on Treatment Effects after Selection among High-Dimensional ControlsReview of Economic Studies, 2013
- Reconstruction From Anisotropic Random MeasurementsIEEE Transactions on Information Theory, 2013
- Forecasting economic time series using targeted predictorsJournal of Econometrics, 2008
- CAN ONE ESTIMATE THE UNCONDITIONAL DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS?Econometric Theory, 2007
- On sparse reconstruction from Fourier and Gaussian measurementsCommunications on Pure and Applied Mathematics, 2007
- Greed is Good: Algorithmic Results for Sparse ApproximationIEEE Transactions on Information Theory, 2004
- Micro-Level Estimation of Poverty and InequalityEconometrica, 2003
- Regression Shrinkage and Selection Via the LassoJournal of the Royal Statistical Society Series B: Statistical Methodology, 1996