Local Rademacher complexities and oracle inequalities in risk minimization
Author:
- Vladimir KoltchinskiiVKVladimir Koltchinskii
Open Access
- 1 December 2006
- journal article
- research article
- Published by Institute of Mathematical Statistics in The Annals of Statistics
- Vol. 34 (6) , 2593-2656
Abstract
Let ℱ be a class of measurable functions f:S↦[0, 1] defined on a probability space (S, , P). Given a sample (X1, …, Xn) of i.i.d. random variables taking values in S with common distribution P, let Pn denote the empirical measure based on (X1, …, Xn). We study an empirical risk minimization problem Pnf→min , f∈ℱ. Given a solution f̂n of this problem, the goal is to obtain very general upper bounds on its excess risk expressed in terms of relevant geometric parameters of the class ℱ. Using concentration inequalities and other empirical processes tools, we obtain both distribution-dependent and data-dependent upper bounds on the excess risk that are of asymptotically correct order in many examples. The bounds involve localized sup-norms of empirical and Rademacher processes indexed by functions from the class. We use these bounds to develop model selection techniques in abstract risk minimization problems that can be applied to more specialized frameworks of regression and classification.Keywords
All Related Versions
This publication has 46 references indexed in Scilit:
- Statistical performance of support vector machinesThe Annals of Statistics, 2008
- Convexity, Classification, and Risk BoundsJournal of the American Statistical Association, 2006
- Empirical minimizationProbability Theory and Related Fields, 2005
- Complexities of convex combinations and bounding the generalization error in classificationThe Annals of Statistics, 2005
- Optimal aggregation of classifiers in statistical learningThe Annals of Statistics, 2004
- Bounding the generalization error of convex combinations of classifiers: balancing the dimensionality and the marginsThe Annals of Applied Probability, 2003
- On consistency of kernel density estimators for randomly censored data: rates holding uniformly over adaptive intervalsAnnales de l'Institut Henri Poincaré, Probabilités et Statistiques, 2001
- Some applications of concentration inequalities to statisticsAnnales de la Faculté des sciences de Toulouse : Mathématiques, 2000
- A new look at independenceThe Annals of Probability, 1996
- Convergence Rate of Sieve EstimatesThe Annals of Statistics, 1994