Kullback–Leibler aggregation and misspecified generalized linear models
Author:
- Philippe RigolletPRPhilippe Rigollet
Open Access
- 1 April 2012
- journal article
- research article
- Published by Institute of Mathematical Statistics in The Annals of Statistics
- Vol. 40 (2) , 639-665
Abstract
In a regression setup with deterministic design, we study the pure aggregation problem and introduce a natural extension from the Gaussian distribution to distributions in the exponential family. While this extension bears strong connections with generalized linear models, it does not require identifiability of the parameter or even that the model on the systematic component is true. It is shown that this problem can be solved by constrained and/or penalized likelihood maximization and we derive sharp oracle inequalities that hold both in expectation and with high probability. Finally all the bounds are proved to be optimal in a minimax sense.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Sparse Estimation by Exponential WeightingStatistical Science, 2012
- PAC-Bayesian bounds for sparse regression estimation with exponential weightsElectronic Journal of Statistics, 2011
- General oracle inequalities for model selectionElectronic Journal of Statistics, 2009
- Learning by mirror averagingThe Annals of Statistics, 2008
- Linear and convex aggregation of density estimatorsMathematical Methods of Statistics, 2007
- Generalized mirror averaging and D-convex aggregationMathematical Methods of Statistics, 2007
- Aggregation for Gaussian regressionThe Annals of Statistics, 2007
- Simultaneous adaptation to the margin and to complexity in classificationThe Annals of Statistics, 2007
- Aggregating regression procedures to improve performanceBernoulli, 2004
- Functional aggregation for nonparametric regressionThe Annals of Statistics, 2000