Testing-Based Forward Model Selection
Author:
Preprint
Abstract
This work introduces a theoretical foundation for a procedure called `testing-based forward model selection' in regression problems. Forward selection is a general term refering to a model selection procedure which inductively selects covariates that add predictive power into a working statistical model. This paper considers the use of testing procedures, derived from traditional statistical hypothesis testing, as a criterion for deciding which variable to include next and when to stop including variables. Probabilistic bounds for prediction error and number of selected covariates are proved for the proposed procedure. The general result is illustrated by an example with heteroskedastic data where Huber-Eicker-White standard errors are used to construct tests. The performance of the testing-based forward selection is compared to Lasso and Post-Lasso in simulation studies. Finally, the use of testing-based forward selection is illustrated with an application to estimating the effects of institution quality on aggregate economic output.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Inference in High-Dimensional Panel Models With an Application to Gun ControlJournal of Business & Economic Statistics, 2016
- Community detection in sparse networks via Grothendieck’s inequalityProbability Theory and Related Fields, 2015
- Instrumental variables estimation with many weak instruments using regularized JIVEJournal of Econometrics, 2014
- Reconstruction From Anisotropic Random MeasurementsIEEE Transactions on Information Theory, 2013
- Grothendieck’s Theorem, past and presentBulletin of the American Mathematical Society, 2012
- Adaptive Forward-Backward Greedy Algorithm for Learning Sparse RepresentationsIEEE Transactions on Information Theory, 2011
- Boosting diffusion indicesJournal of Applied Econometrics, 2009
- Forecasting economic time series using targeted predictorsJournal of Econometrics, 2008
- SHRINKAGE ESTIMATION FOR NEARLY SINGULAR DESIGNSEconometric Theory, 2007
- On sparse reconstruction from Fourier and Gaussian measurementsCommunications on Pure and Applied Mathematics, 2007