Remember to Correct the Bias When Using Deep Learning for Regression!
Open Access
- 1 March 2023
- journal article
- research article
- Published by Springer Nature in KI - Künstliche Intelligenz
- Vol. 37 (1) , 33-40
Abstract
When training deep learning models for least-squares regression, we cannot expect that the training error residuals of the final model, selected after a fixed training time or based on performance on a hold-out data set, sum to zero. This can introduce a systematic error that accumulates if we are interested in the total aggregated performance over many data points (e.g., the sum of the residuals on previously unseen data). We suggest adjusting the bias of the machine learning model after training as a default post-processing step, which efficiently solves the problem. The severeness of the error accumulation and the effectiveness of the bias correction are demonstrated in exemplary experiments.Keywords
All Related Versions
Funding Information
- Villum Fonden (Deep Learning and Remote Sensing for Unlocking Global Ecosystem Resource Dynamics (DeReEco))
- Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI)
This publication has 18 references indexed in Scilit:
- Predicting CO and NOxemissions from gas turbines: novel data and abenchmark PEMSTurkish Journal of Electrical Engineering and Computer Sciences, 2019
- MnasNet: Platform-Aware Neural Architecture Search for Mobile2019
- Squeeze-and-Excitation Networks2018
- Allometric equations for integrating remote sensing imagery into forest monitoring programmesGlobal Change Biology, 2016
- Early Stopping — But When?2012
- Bias Correction in Logarithmic Regression and Comparison with Weighted Regression for Nonlinear ModelsNature Precedings, 2011
- QSAR Modeling Using Automatically Updating Correction Libraries: Application to a Human Plasma Protein Binding ModelJournal of Chemical Information and Modeling, 2007
- logD7.4 Modeling Using Bayesian Regularized Neural Networks. Assessment and Correction of the Errors of PredictionJournal of Chemical Information and Modeling, 2005
- Use of Logarithmic Regression in the Estimation of Plant BiomassCanadian Journal of Forest Research, 1972
- On the Distribution of a Variate Whose Logarithm is Normally DistributedJournal of the Royal Statistical Society Series B: Statistical Methodology, 1941