Global quantitative robustness of regression feed-forward neural networks
Author:
Open Access
- 9 August 2024
- journal article
- research article
- Published by Springer Nature in Neural Computing and Applications
- Vol. 36 (32) , 19967-19988
Abstract
Neural networks are an indispensable model class for many complex learning tasks. Despite the popularity and importance of neural networks and many different established techniques from literature for stabilization and robustification of the training, the classical concepts from robust statistics have rarely been considered so far in the context of neural networks. Therefore, we adapt the notion of the regression breakdown point to regression neural networks and compute the breakdown point for different feed-forward network configurations and contamination settings. In an extensive simulation study, we compare the performance, measured by the out-of-sample loss, by a proxy of the breakdown rate and by the training steps, of non-robust and robust regression feed-forward neural networks in a plethora of different configurations. The results indeed motivate to use robust loss functions for neural network training.Keywords
All Related Versions
Funding Information
- Carl von Ossietzky Universität Oldenburg
This publication has 43 references indexed in Scilit:
- A robust training of dendritic neuron model neural network for time series predictionNeural Computing and Applications, 2023
- Regression Neural Networks with a Highly Robust Loss Function2020
- Reducing noise impact on MLP trainingSoft Computing, 2015
- Training Neural Networks on Noisy DataLecture Notes in Computer Science, 2014
- Robust Statistics2009
- Robust Training of Artificial Feedforward Neural Networks2009
- Robust LTS Backpropagation Learning Algorithm2007
- Computing LTS Regression for Large Data SetsData Mining and Knowledge Discovery, 2006
- An Algorithm for Positive-Breakdown Regression Based on Concentration Steps2000
- Robust Asymptotic Statistics1994