Dimensionality Reduction, Regularization, and Generalization in Overparameterized Regressions
Authors:
- Ningyuan TeresaNTNingyuan Teresa
- David W. HoggDHDavid W. Hogg
- Soledad VillarSVSoledad Villar
- 10 February 2022
- journal article
- research article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Mathematics of Data Science
- Vol. 4 (1) , 126-152
Abstract
Overparameterization in deep learning is powerful: Very large models fit the training data perfectly and yet often generalize well. This realization brought back the study of linear models for regr...Keywords
All Related Versions
Funding Information
- European Office of Aerospace Research and Development (FA9550-18-1-7007)
- NSF-Simons Resarch Collaboration on the Mathematical and Scientific Foundations of Deep Learning (DMS 2031985)
- National Science Foundation (DMS 2044349)
This publication has 31 references indexed in Scilit:
- Concentration inequalities and moment bounds for sample covariance operatorsBernoulli, 2017
- THE CANNON: A DATA-DRIVEN APPROACH TO STELLAR LABEL DETERMINATIONThe Astrophysical Journal, 2015
- A Paradoxical Result in Estimating Regression CoefficientsThe American Statistician, 2014
- Robust PCA via Outlier PursuitIEEE Transactions on Information Theory, 2012
- A paradox concerning nuisance parameters and projected estimating functionsBiometrika, 2004
- Statistical pattern recognition: a reviewIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
- Comparison of Prediction Methods when Only a Few Components are RelevantJournal of the American Statistical Association, 1994
- A Statistical View of Some Chemometrics Regression ToolsTechnometrics, 1993
- On the interlacing property for singular values and eigenvaluesLinear Algebra and its Applications, 1987
- Principal Components Regression in Exploratory Statistical ResearchJournal of the American Statistical Association, 1965