Ernesto De Vito
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Ernesto De Vito has a total of 96 co-authors. Here are the top co-authors by number of shared publications:
- Filippo De Mari
University of Genoa
19 shared publications - Lorenzo Rosasco
Malga -DIBRIS University of Genova IT
17 shared publications - Alessandro Toigo
Dipartimento di Matematica, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milano, Italy
12 shared publications G. Cassinelli
Dipartimento di Fisica, Università di Genova, and I.N.F.N., Sezione di Genova, Via Dodecaneso 33, 16146 Genova, Italy
10 shared publicationsA. Levrero
7 shared publications- Francesca Bartolucci
Institute of Applied Mathematics TU Delft Mekelweg 4 2628 CD Delft The Netherlands
7 shared publications
Latest Publications
- Open Access
On Coorbit Fréchet Spaces
This paper is concerned with a new approach to coorbit space theory. Usually, coorbit spaces are defined by collecting all distributions for which the voice transform associated with a square-integrable group representation possesses a certain decay, usually measured in a Banach space norm such as weighted $L_p$-norms. Unfortunately, in cases where the representation does not satisfy certain integrability conditions, one is faced with a bottleneck, namely that the discretization of the coorbit spaces is surprisingly difficult. It turns out that in these cases the construction of coorbit spaces as Fréchet spaces is much more convenient since then atomic decompositions can be established in a very natural way.
Open AccessOn Coorbit Fréchet Spaces
This paper is concerned with a new approach to coorbit space theory. Usually, coorbit spaces are defined by collecting all distributions for which the voice transform associated with a square-integrable group representation possesses a certain decay, usually measured in a Banach space norm such as weighted $L_p$-norms. Unfortunately, in cases where the representation does not satisfy certain integrability conditions, one is faced with a bottleneck, namely that the discretization of the coorbit spaces is surprisingly difficult. It turns out that in these cases the construction of coorbit spaces as Fréchet spaces is much more convenient since then atomic decompositions can be established in a very natural way.
- Open Access
Learning Multi-Index Models with Hyper-Kernel Ridge Regression
Deep neural networks excel in high-dimensional problems, outperforming models such as kernel methods, which suffer from the curse of dimensionality. However, the theoretical foundations of this success remain poorly understood. We follow the idea that the compositional structure of the learning task is the key factor determining when deep networks outperform other approaches. Taking a step towards formalizing this idea, we consider a simple compositional model, namely the multi-index model (MIM). In this context, we introduce and study hyper-kernel ridge regression (HKRR), an approach blending neural networks and kernel methods. Our main contribution is a sample complexity result demonstrating that HKRR can adaptively learn MIM, overcoming the curse of dimensionality. Further, we exploit the kernel nature of the estimator to develop ad hoc optimization approaches. Indeed, we contrast alternating minimization and alternating gradient methods both theoretically and numerically. These numerical results complement and reinforce our theoretical findings.
Open AccessLearning Multi-Index Models with Hyper-Kernel Ridge Regression
Deep neural networks excel in high-dimensional problems, outperforming models such as kernel methods, which suffer from the curse of dimensionality. However, the theoretical foundations of this success remain poorly understood. We follow the idea that the compositional structure of the learning task is the key factor determining when deep networks outperform other approaches. Taking a step towards formalizing this idea, we consider a simple compositional model, namely the multi-index model (MIM). In this context, we introduce and study hyper-kernel ridge regression (HKRR), an approach blending neural networks and kernel methods. Our main contribution is a sample complexity result demonstrating that HKRR can adaptively learn MIM, overcoming the curse of dimensionality. Further, we exploit the kernel nature of the estimator to develop ad hoc optimization approaches. Indeed, we contrast alternating minimization and alternating gradient methods both theoretically and numerically. These numerical results complement and reinforce our theoretical findings.
- Open Access
Computational Efficiency under Covariate Shift in Kernel Ridge Regression
This paper addresses the covariate shift problem in the context of nonparametric regression within reproducing kernel Hilbert spaces (RKHSs). Covariate shift arises in supervised learning when the input distributions of the training and test data differ, presenting additional challenges for learning. Although kernel methods have optimal statistical properties, their high computational demands in terms of time and, particularly, memory, limit their scalability to large datasets. To address this limitation, the main focus of this paper is to explore the trade-off between computational efficiency and statistical accuracy under covariate shift. We investigate the use of random projections where the hypothesis space consists of a random subspace within a given RKHS. Our results show that, even in the presence of covariate shift, significant computational savings can be achieved without compromising learning performance.
Open AccessComputational Efficiency under Covariate Shift in Kernel Ridge Regression
This paper addresses the covariate shift problem in the context of nonparametric regression within reproducing kernel Hilbert spaces (RKHSs). Covariate shift arises in supervised learning when the input distributions of the training and test data differ, presenting additional challenges for learning. Although kernel methods have optimal statistical properties, their high computational demands in terms of time and, particularly, memory, limit their scalability to large datasets. To address this limitation, the main focus of this paper is to explore the trade-off between computational efficiency and statistical accuracy under covariate shift. We investigate the use of random projections where the hypothesis space consists of a random subspace within a given RKHS. Our results show that, even in the presence of covariate shift, significant computational savings can be achieved without compromising learning performance.
- Open Access
- 4 November 2024
- Published by IOP Publishing in Inverse Problems
- Vol. 40 (12) , 125004
On learning the optimal regularization parameter in inverse problems
Selecting the best regularization parameter in inverse problems is a classical and yet challenging problem. Recently, data-driven approaches based on supervised learning have become popular to tackle this challenge. These approaches are appealing since they do require less a priori knowledge, but their theoretical analysis is limited. In this paper, we propose and study a statistical machine learning approach, based on empirical risk minimization. Our main contribution is a theoretical analysis, showing that, provided with enough data, this approach can reach sharp rates while being essentially adaptive to the noise and smoothness of the problem. Numerical simulations corroborate and illustrate the theoretical findings. Our results are a step towards grounding theoretically data-driven approaches based on supervised learning to inverse problems.
- 4 November 2024
- Published by IOP Publishing in Inverse Problems
- Vol. 40 (12) , 125004
Open AccessOn learning the optimal regularization parameter in inverse problems
Selecting the best regularization parameter in inverse problems is a classical and yet challenging problem. Recently, data-driven approaches based on supervised learning have become popular to tackle this challenge. These approaches are appealing since they do require less a priori knowledge, but their theoretical analysis is limited. In this paper, we propose and study a statistical machine learning approach, based on empirical risk minimization. Our main contribution is a theoretical analysis, showing that, provided with enough data, this approach can reach sharp rates while being essentially adaptive to the noise and smoothness of the problem. Numerical simulations corroborate and illustrate the theoretical findings. Our results are a step towards grounding theoretically data-driven approaches based on supervised learning to inverse problems.
- 5 September 2022
- Published by Elsevier in Applied and Computational Harmonic Analysis
- Vol. 62, 194-236
Understanding neural networks with reproducing kernel Banach spaces
- 5 September 2022
- Published by Elsevier in Applied and Computational Harmonic Analysis
- Vol. 62, 194-236
Understanding neural networks with reproducing kernel Banach spaces
- 1 January 2022
- Published by Elsevier in Applied and Computational Harmonic Analysis
- Vol. 56, 123-149
Continuous wavelet frames on the sphere: The group-theoretic approach revisited
- 1 January 2022
- Published by Elsevier in Applied and Computational Harmonic Analysis
- Vol. 56, 123-149
Continuous wavelet frames on the sphere: The group-theoretic approach revisited
- 14 December 2021
- Published by Springer Nature
- p. 245-296
Regularization: From Inverse Problems to Large-Scale Machine Learning
- 14 December 2021
- Published by Springer Nature
- p. 245-296
Regularization: From Inverse Problems to Large-Scale Machine Learning
- Open Access
Learning the optimal Tikhonov regularizer for inverse problems
In this work, we consider the linear inverse problem $y=Ax+\epsilon$, where $A\colon X\to Y$ is a known linear operator between the separable Hilbert spaces $X$ and $Y$, $x$ is a random variable in $X$ and $\epsilon$ is a zero-mean random process in $Y$. This setting covers several inverse problems in imaging including denoising, deblurring, and X-ray tomography. Within the classical framework of regularization, we focus on the case where the regularization functional is not given a priori but learned from data. Our first result is a characterization of the optimal generalized Tikhonov regularizer, with respect to the mean squared error. We find that it is completely independent of the forward operator $A$ and depends only on the mean and covariance of $x$. Then, we consider the problem of learning the regularizer from a finite training set in two different frameworks: one supervised, based on samples of both $x$ and $y$, and one unsupervised, based only on samples of $x$. In both cases, we prove generalization bounds, under some weak assumptions on the distribution of $x$ and $\epsilon$, including the case of sub-Gaussian variables. Our bounds hold in infinite-dimensional spaces, thereby showing that finer and finer discretizations do not make this learning problem harder. The results are validated through numerical simulations.Open AccessLearning the optimal Tikhonov regularizer for inverse problems
In this work, we consider the linear inverse problem $y=Ax+\epsilon$, where $A\colon X\to Y$ is a known linear operator between the separable Hilbert spaces $X$ and $Y$, $x$ is a random variable in $X$ and $\epsilon$ is a zero-mean random process in $Y$. This setting covers several inverse problems in imaging including denoising, deblurring, and X-ray tomography. Within the classical framework of regularization, we focus on the case where the regularization functional is not given a priori but learned from data. Our first result is a characterization of the optimal generalized Tikhonov regularizer, with respect to the mean squared error. We find that it is completely independent of the forward operator $A$ and depends only on the mean and covariance of $x$. Then, we consider the problem of learning the regularizer from a finite training set in two different frameworks: one supervised, based on samples of both $x$ and $y$, and one unsupervised, based only on samples of $x$. In both cases, we prove generalization bounds, under some weak assumptions on the distribution of $x$ and $\epsilon$, including the case of sub-Gaussian variables. Our bounds hold in infinite-dimensional spaces, thereby showing that finer and finer discretizations do not make this learning problem harder. The results are validated through numerical simulations. - Open Access
Understanding neural networks with reproducing kernel Banach spaces
Characterizing the function spaces corresponding to neural networks can provide a way to understand their properties. In this paper we discuss how the theory of reproducing kernel Banach spaces can be used to tackle this challenge. In particular, we prove a representer theorem for a wide class of reproducing kernel Banach spaces that admit a suitable integral representation and include one hidden layer neural networks of possibly infinite width. Further, we show that, for a suitable class of ReLU activation functions, the norm in the corresponding reproducing kernel Banach space can be characterized in terms of the inverse Radon transform of a bounded real measure, with norm given by the total variation norm of the measure. Our analysis simplifies and extends recent results in [34,29,30].
Open AccessUnderstanding neural networks with reproducing kernel Banach spaces
Characterizing the function spaces corresponding to neural networks can provide a way to understand their properties. In this paper we discuss how the theory of reproducing kernel Banach spaces can be used to tackle this challenge. In particular, we prove a representer theorem for a wide class of reproducing kernel Banach spaces that admit a suitable integral representation and include one hidden layer neural networks of possibly infinite width. Further, we show that, for a suitable class of ReLU activation functions, the norm in the corresponding reproducing kernel Banach space can be characterized in terms of the inverse Radon transform of a bounded real measure, with norm given by the total variation norm of the measure. Our analysis simplifies and extends recent results in [34,29,30].
- Open Access
- 16 April 2021
- Published by Springer Nature in Journal of Fourier Analysis and Applications
- Vol. 27 (2) , 1-39
Construction and Monte Carlo Estimation of Wavelet Frames Generated by a Reproducing Kernel
We introduce a construction of multiscale tight frames on general domains. The frame elements are obtained by spectral filtering of the integral operator associated with a reproducing kernel. Our construction extends classical wavelets as well as generalized wavelets on both continuous and discrete non-Euclidean structures such as Riemannian manifolds and weighted graphs. Moreover, it allows to study the relation between continuous and discrete frames in a random sampling regime, where discrete frames can be seen as Monte Carlo estimates of the continuous ones. Pairing spectral regularization with learning theory, we show that a sample frame tends to its population counterpart, and derive explicit finite-sample rates on spaces of Sobolev and Besov regularity. Our results prove the stability of frames constructed on empirical data, in the sense that all stochastic discretizations have the same underlying limit regardless of the set of initial training samples.
- 16 April 2021
- Published by Springer Nature in Journal of Fourier Analysis and Applications
- Vol. 27 (2) , 1-39
Open AccessConstruction and Monte Carlo Estimation of Wavelet Frames Generated by a Reproducing Kernel
We introduce a construction of multiscale tight frames on general domains. The frame elements are obtained by spectral filtering of the integral operator associated with a reproducing kernel. Our construction extends classical wavelets as well as generalized wavelets on both continuous and discrete non-Euclidean structures such as Riemannian manifolds and weighted graphs. Moreover, it allows to study the relation between continuous and discrete frames in a random sampling regime, where discrete frames can be seen as Monte Carlo estimates of the continuous ones. Pairing spectral regularization with learning theory, we show that a sample frame tends to its population counterpart, and derive explicit finite-sample rates on spaces of Sobolev and Besov regularity. Our results prove the stability of frames constructed on empirical data, in the sense that all stochastic discretizations have the same underlying limit regardless of the set of initial training samples.