Improving Predictions by Nonlinear Regression Models from Outlying Input Data
Author:
Open Access
- 1 January 2023
- journal article
- research article
- Published by International Society for Environmental Information Science (ISEIS) in Journal of Environmental Informatics
- Vol. 41 (2) , 75-87-87
Abstract
When applying machine learning/statistical methods to the environmental sciences, nonlinear regression (NLR) models often perform only slightly better and occasionally worse than linear regression (LR). The proposed reason for this conundrum is that NLR models can give predictions much worse than LR when given input data which lie outside the domain used in model training. Continuous unbounded variables are widely used in environmental sciences, whence not uncommon for new input data to lie far outside the training domain. For six environmental datasets, inputs in the test data were classified as “outliers” and “non-outliers” based on the Mahalanobis distance from the training input data. The prediction scores (mean absolute error, Spearman correlation) showed NLR to outperform LR for the non-outliers, but often underperform LR for the outliers. An approach based on Occam’s Razor (OR) was proposed, where linear extrapolation was used instead of nonlinear extrapolation for the outliers. The linear extrapolation to the outlier domain was based on the NLR model within the non-outlier domain. This NLROR approach reduced occurrences of very poor extrapolation by NLR, and it tended to outperform NLR and LR for the outliers. In conclusion, input test data should be screened for outliers. For outliers, the unreliable NLR predictions can be replaced by NLROR or LR predictions, or by issuing a “no reliable prediction” warning.Keywords
This publication has 38 references indexed in Scilit:
- Linear and nonlinear regression prediction of surface wind componentsClimate Dynamics, 2018
- PM2.5data reliability, consistency, and air quality assessment in five Chinese citiesJournal of Geophysical Research: Atmospheres, 2016
- Evaluating hourly air quality forecasting in Canada with nonlinear updatable machine learning methodsAir Quality, Atmosphere & Health, 2016
- The probability distribution of intense daily precipitationGeophysical Research Letters, 2015
- An Insight into Extreme Learning Machines: Random Neurons, Random Features and KernelsCognitive Computation, 2014
- Comparison of statistically downscaled precipitation in terms of future climate indices and daily variability for southern Ontario and Quebec, CanadaClimate Dynamics, 2014
- Extreme learning machines: a surveyInternational Journal of Machine Learning and Cybernetics, 2011
- The impact of time‐averaging on the detectability of nonlinear empirical relationsQuarterly Journal of the Royal Meteorological Society, 2002
- Support-vector networksMachine Learning, 1995
- Diagnosis using backpropagation neural networks—analysis and criticismComputers & Chemical Engineering, 1990