Learning Korobov Functions by Correntropy and Convolutional Neural Networks
- 21 March 2024
- journal article
- research article
- Published by MIT Press in Neural Computation
- Vol. 36 (4) , 718-743
Abstract
Combining information-theoretic learning with deep learning has gained significant attention in recent years, as it offers a promising approach to tackle the challenges posed by big data. However, the theoretical understanding of convolutional structures, which are vital to many structured deep learning models, remains incomplete. To partially bridge this gap, this letter aims to develop generalization analysis for deep convolutional neural network (CNN) algorithms using learning theory. Specifically, we focus on investigating robust regression using correntropy-induced loss functions derived from information-theoretic learning. Our analysis demonstrates an explicit convergence rate for deep CNN-based robust regression algorithms when the target function resides in the Korobov space. This study sheds light on the theoretical underpinnings of CNNs and provides a framework for understanding their performance and limitations.Keywords
This publication has 26 references indexed in Scilit:
- Learning ability of interpolating deep convolutional neural networksApplied and Computational Harmonic Analysis, 2023
- Approximation of Nonlinear Functionals Using Deep ReLU NetworksJournal of Fourier Analysis and Applications, 2023
- Optimality of Robust Online LearningFoundations of Computational Mathematics, 2023
- Approximation of functions from Korobov spaces by deep convolutional neural networksAdvances in Computational Mathematics, 2022
- Learning under (1 + ϵ)-moment conditionsApplied and Computational Harmonic Analysis, 2020
- Learning with correntropy-induced losses for regression with mixture of symmetric stable noiseApplied and Computational Harmonic Analysis, 2020
- Universality of deep convolutional neural networksApplied and Computational Harmonic Analysis, 2020
- Optimal learning rates for distribution regressionJournal of Complexity, 2020
- Information Theoretic Learning2010
- Limitations of the approximation capabilities of neural networks with one hidden layerAdvances in Computational Mathematics, 1996