Olivier Catoni
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Olivier Catoni has a total of 2 co-authors. Here are the top co-authors by number of shared publications:
- Jean-Yves Audibert
Université Paris-Est LIGM Imagine 6 avenue Blaise Pascal 77455 Marne-la-Vallée France
2 shared publications - Raphaël Cerf
Université Paris Sud and CNRS, Mathématique, Bâtiment 425, 91405 Orsay Cedex, France.¶; FR
1 shared publication
Latest Publications
- 4 September 2015
- Published by Springer Nature
- p. 157-160
Comment: Transductive PAC-Bayes Bounds Seen as a Generalization of Vapnik–Chervonenkis Bounds
- 4 September 2015
- Published by Springer Nature
- p. 157-160
Comment: Transductive PAC-Bayes Bounds Seen as a Generalization of Vapnik–Chervonenkis Bounds
- 4 September 2015
- Published by Springer Nature
- p. 287-302
PAC-Bayes Bounds for Supervised Classification
- 4 September 2015
- Published by Springer Nature
- p. 287-302
PAC-Bayes Bounds for Supervised Classification
- Open Access
Linear regression through PAC-Bayesian truncation
We consider the problem of predicting as well as the best linear combination of d given functions in least squares regression under L^\infty constraints on the linear combination. When the input distribution is known, there already exists an algorithm having an expected excess risk of order d/n, where n is the size of the training data. Without this strong assumption, standard results often contain a multiplicative log(n) factor, complex constants involving the conditioning of the Gram matrix of the covariates, kurtosis coefficients or some geometric quantity characterizing the relation between L^2 and L^\infty-balls and require some additional assumptions like exponential moments of the output. This work provides a PAC-Bayesian shrinkage procedure with a simple excess risk bound of order d/n holding in expectation and in deviations, under various assumptions. The common surprising factor of these results is their simplicity and the absence of exponential moment condition on the output distribution while achieving exponential deviations. The risk bounds are obtained through a PAC-Bayesian analysis on truncated differences of losses. We also show that these results can be generalized to other strongly convex loss functions.
Open AccessLinear regression through PAC-Bayesian truncation
We consider the problem of predicting as well as the best linear combination of d given functions in least squares regression under L^\infty constraints on the linear combination. When the input distribution is known, there already exists an algorithm having an expected excess risk of order d/n, where n is the size of the training data. Without this strong assumption, standard results often contain a multiplicative log(n) factor, complex constants involving the conditioning of the Gram matrix of the covariates, kurtosis coefficients or some geometric quantity characterizing the relation between L^2 and L^\infty-balls and require some additional assumptions like exponential moments of the output. This work provides a PAC-Bayesian shrinkage procedure with a simple excess risk bound of order d/n holding in expectation and in deviations, under various assumptions. The common surprising factor of these results is their simplicity and the absence of exponential moment condition on the output distribution while achieving exponential deviations. The risk bounds are obtained through a PAC-Bayesian analysis on truncated differences of losses. We also show that these results can be generalized to other strongly convex loss functions.
- Open Access
Risk bounds in linear regression through PAC-Bayesian truncation
We consider the problem of predicting as well as the best linear combination of d given functions in least squares regression, and variants of this problem including constraints on the parameters of the linear combination. When the input distribution is known, there already exists an algorithm having an expected excess risk of order d/n, where n is the size of the training data. Without this strong assumption, standard results often contain a multiplicative log n factor, and require some additional assumptions like uniform boundedness of the d-dimensional input representation and exponential moments of the output. This work provides new risk bounds for the ridge estimator and the ordinary least squares estimator, and their variants. It also provides shrinkage procedures with convergence rate d/n (i.e., without the logarithmic factor) in expectation and in deviations, under various assumptions. The key common surprising factor of these results is the absence of exponential moment condition on the output distribution while achieving exponential deviations. All risk bounds are obtained through a PAC-Bayesian analysis on truncated differences of losses. Finally, we show that some of these results are not particular to the least squares loss, but can be generalized to similar strongly convex loss functions.
Open AccessRisk bounds in linear regression through PAC-Bayesian truncation
We consider the problem of predicting as well as the best linear combination of d given functions in least squares regression, and variants of this problem including constraints on the parameters of the linear combination. When the input distribution is known, there already exists an algorithm having an expected excess risk of order d/n, where n is the size of the training data. Without this strong assumption, standard results often contain a multiplicative log n factor, and require some additional assumptions like uniform boundedness of the d-dimensional input representation and exponential moments of the output. This work provides new risk bounds for the ridge estimator and the ordinary least squares estimator, and their variants. It also provides shrinkage procedures with convergence rate d/n (i.e., without the logarithmic factor) in expectation and in deviations, under various assumptions. The key common surprising factor of these results is the absence of exponential moment condition on the output distribution while achieving exponential deviations. All risk bounds are obtained through a PAC-Bayesian analysis on truncated differences of losses. Finally, we show that some of these results are not particular to the least squares loss, but can be generalized to similar strongly convex loss functions.
- Open Access
- 1 January 1997
- Published by EDP Sciences in ESAIM: Probability and Statistics
- Vol. 1, 95-144
The exit path of a Markov chain with rare transitions
We study the exit path from a general domain after the last visit to a set of a Markov chain with rare transitions. We prove several large deviation principles for the law of the succession of the cycles visited by the process (the cycle path), the succession of the saddle points gone through to jump from cycle to cycle on the cycle path (the saddle path) and the succession of all the points gone through (the exit path). We estimate the time the process spends in each cycle of the cycle path and how it decomposes into the time spent in each point of the exit path. We describe a systematic method to find the most likely saddle paths. We apply these results to the reversible case of the Metropolis dynamics. We give in appendix the corresponding large deviation estimates in the non homogeneous case, which are corollaries of already published works by Catoni (1992) and Trouvé (1992, 1996a). Nous étudions le chemin de sortie d'une chaîne de Markov à transitions rares après son dernier passage par un sous domaine d'un domaine quelconque. Nous établissons des estimées de grandes déviations pour la suite des cycles visités par le processus, la suite des points selle qu'il emprunte et la suite de tous les états constituant le support du chemin de sortie. Nous estimons aussi le temps passé en chaque cycle et sa répartition entre ses différents états. Nous décrivons une méthode systématique pour déterminer les suites de points selle les plus probables, et traitons plus en détail le cas particulier des dynamiques de Metropolis réversibles. Nous donnons en appendice des estimées de grandes déviations correspondant au cas plus général des chaînes non homogènes, qui s'obtiennent comme corollaires de travaux antérieurs de Catoni (1992) et Trouvé (1992, 1996a).
- 1 January 1997
- Published by EDP Sciences in ESAIM: Probability and Statistics
- Vol. 1, 95-144
Open AccessThe exit path of a Markov chain with rare transitions
We study the exit path from a general domain after the last visit to a set of a Markov chain with rare transitions. We prove several large deviation principles for the law of the succession of the cycles visited by the process (the cycle path), the succession of the saddle points gone through to jump from cycle to cycle on the cycle path (the saddle path) and the succession of all the points gone through (the exit path). We estimate the time the process spends in each cycle of the cycle path and how it decomposes into the time spent in each point of the exit path. We describe a systematic method to find the most likely saddle paths. We apply these results to the reversible case of the Metropolis dynamics. We give in appendix the corresponding large deviation estimates in the non homogeneous case, which are corollaries of already published works by Catoni (1992) and Trouvé (1992, 1996a). Nous étudions le chemin de sortie d'une chaîne de Markov à transitions rares après son dernier passage par un sous domaine d'un domaine quelconque. Nous établissons des estimées de grandes déviations pour la suite des cycles visités par le processus, la suite des points selle qu'il emprunte et la suite de tous les états constituant le support du chemin de sortie. Nous estimons aussi le temps passé en chaque cycle et sa répartition entre ses différents états. Nous décrivons une méthode systématique pour déterminer les suites de points selle les plus probables, et traitons plus en détail le cas particulier des dynamiques de Metropolis réversibles. Nous donnons en appendice des estimées de grandes déviations correspondant au cas plus général des chaînes non homogènes, qui s'obtiennent comme corollaires de travaux antérieurs de Catoni (1992) et Trouvé (1992, 1996a).