Mirror Descent on Reproducing Kernel Banach Spaces
Preprint
Abstract
Recent advances in machine learning have led to increased interest in reproducing kernel Banach spaces (RKBS) as a more general framework that extends beyond reproducing kernel Hilbert spaces (RKHS). These works have resulted in the formulation of representer theorems under several regularized learning schemes. However, little is known about an optimization method that encompasses these results in this setting. This paper addresses a learning problem on Banach spaces endowed with a reproducing kernel, focusing on efficient optimization within RKBS. To tackle this challenge, we propose an algorithm based on mirror descent (MDA). Our approach involves an iterative method that employs gradient steps in the dual space of the Banach space using the reproducing kernel. We analyze the convergence properties of our algorithm under various assumptions and establish two types of results: first, we identify conditions under which a linear convergence rate is achievable, akin to optimization in the Euclidean setting, and provide a proof of the linear rate; second, we demonstrate a standard convergence rate in a constrained setting. Moreover, to instantiate this algorithm in practice, we introduce a novel family of RKBSs with $p$-norm ($p \neq 2$), characterized by both an explicit dual map and a kernel.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- On Reproducing Kernel Banach Spaces: Generic Definitions and Unified Framework of ConstructionsActa Mathematica Sinica, English Series, 2022
- Approximation Spaces of Deep Neural NetworksConstructive Approximation, 2021
- Solving support vector machines in reproducing kernel Banach spaces with positive definite functionsApplied and Computational Harmonic Analysis, 2015
- Reproducing kernel Banach spaces with the ℓ1 normApplied and Computational Harmonic Analysis, 2013
- Concentration estimates for learning with-regularizer and data dependent hypothesis spacesApplied and Computational Harmonic Analysis, 2011
- Frames, Riesz bases, and sampling expansions in Banach spaces via semi-inner productsApplied and Computational Harmonic Analysis, 2011
- Regularized learning in Banach spaces as an optimization problem: representer theoremsJournal of Global Optimization, 2010
- Feature space perspectives for learning the kernelMachine Learning, 2007
- Proximal minimization algorithm withD-functionsJournal of Optimization Theory and Applications, 1992
- Gradient methods for the minimisation of functionalsUSSR Computational Mathematics and Mathematical Physics, 1963