The estimation problem of minimum mean squared error
Authors:
- Luc DevroyeLDLuc Devroye
- Dominik SchäferDSDominik Schäfer
- László GyörfiLGLászló Györfi
- Harro WalkHWHarro Walk
- 1 January 2003
- journal article
- research article
- Published by Walter de Gruyter GmbH in Statistics & Decisions
- Vol. 21 (1-2003) , 15-28
Abstract
Regression analysis of a response variable Y requires careful selection of explanatory variables. The quality of a set of explanatory features X =( X (1) ,..., X ( d ) ) can be measured in terms of the minimum mean squared error . This paper investigates methods for estimating L * from i.i.d. data. No estimate can converge rapidly for all distributions of ( X , Y ). For Lipschitz continuous regression function E { Y | X = x }, two estimators for L * are discussed: fitting a regression estimate to a subset of the data and assessing its mean residual sum of squares on the remaining samples, and a nearest neighbor cross-validation type estimate.Keywords
This publication has 9 references indexed in Scilit:
- Lower bounds for Bayes error estimationIEEE Transactions on Pattern Analysis and Machine Intelligence, 1999
- Nonparametric Recursive Variance EstimationStatistics, 1995
- On the Strong Universal Consistency of Nearest Neighbor Regression Function EstimatesThe Annals of Statistics, 1994
- Fully Data-Driven Nonparametric Variance EstimatorsStatistics, 1994
- An equivalence theorem for L1 convergence of the kernel regression estimateJournal of Statistical Planning and Inference, 1989
- Estimation of Heteroscedasticity in Regression AnalysisThe Annals of Statistics, 1987
- Consistent Window Estimation in Nonparametric RegressionThe Annals of Statistics, 1980
- Distribution-Free Consistency Results in Nonparametric Discrimination and Regression Function EstimationThe Annals of Statistics, 1980
- Consistent Nonparametric RegressionThe Annals of Statistics, 1977