A unifying representer theorem for inverse problems and machine learning
Author:
Preprint
Abstract
The standard approach for dealing with the ill-posedness of the training problem in machine learning and/or the reconstruction of a signal from a limited number of measurements is regularization. The method is applicable whenever the problem is formulated as an optimization task. The standard strategy consists in augmenting the original cost functional by an energy that penalizes solutions with undesirable behavior. The effect of regularization is very well understood when the penalty involves a Hilbertian norm. Another popular configuration is the use of an $\ell_1$-norm (or some variant thereof) that favors sparse solutions. In this paper, we propose a higher-level formulation of regularization within the context of Banach spaces. We present a general representer theorem that characterizes the solutions of a remarkably broad class of optimization problems. We then use our theorem to retrieve a number of known results in the literature---e.g., the celebrated representer theorem of machine leaning for RKHS, Tikhonov regularization, representer theorems for sparsity promoting functionals, the recovery of spikes---as well as a few new ones.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Support Recovery for Sparse Super-Resolution of Positive MeasuresJournal of Fourier Analysis and Applications, 2016
- Exact Support Recovery for Sparse Spikes DeconvolutionFoundations of Computational Mathematics, 2014
- Towards a Mathematical Theory of Super‐resolutionCommunications on Pure and Applied Mathematics, 2013
- A Mathematical Introduction to Compressive Sensing2013
- Semi-reproducing kernel Hilbert spaces, splines and increment krigingJournal of Nonparametric Statistics, 2010
- Irrotational or divergence-free interpolationNumerische Mathematik, 2004
- Spline solutions to L1 extremal problems in one and several variablesJournal of Approximation Theory, 1975
- Some results on Tchebycheffian spline functionsJournal of Mathematical Analysis and Applications, 1971
- A theorem on duality mappings in Banach spacesArkiv för Matematik, 1962
- Sur les fonctions conjugu esMathematische Zeitschrift, 1928