Adaptive Parameter Selection for Kernel Ridge Regression
Author:
Preprint
Abstract
This paper focuses on parameter selection issues of kernel ridge regression (KRR). Due to special spectral properties of KRR, we find that delicate subdivision of the parameter interval shrinks the difference between two successive KRR estimates. Based on this observation, we develop an early-stopping type parameter selection strategy for KRR according to the so-called Lepskii-type principle. Theoretical verifications are presented in the framework of learning theory to show that KRR equipped with the proposed parameter selection strategy succeeds in achieving optimal learning rates and adapts to different norms, providing a new record of parameter selection for kernel methods.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Balancing principle in supervised learning for a general regularization schemeApplied and Computational Harmonic Analysis, 2020
- Total stability of kernel methodsNeurocomputing, 2018
- Learning theory of distributed spectral algorithmsInverse Problems, 2017
- Optimal Rates for Regularization of Statistical Inverse Learning ProblemsFoundations of Computational Mathematics, 2017
- Adaptive Kernel Methods Using the Balancing PrincipleFoundations of Computational Mathematics, 2010
- Learning Theory Estimates via Integral Operators and Their ApproximationsConstructive Approximation, 2007
- Optimal Rates for the Regularized Least-Squares AlgorithmFoundations of Computational Mathematics, 2006
- Shannon sampling II: Connections to learning theoryApplied and Computational Harmonic Analysis, 2005
- Best Choices for Regularization Parameters in Learning Theory: On the Bias—Variance ProblemMathematische Annalen, 2002
- A Distribution-Free Theory of Nonparametric Regression2002