Assessing Objective Functions in Streamflow Prediction Model Training Based on the Naïve Method
Open Access
Abstract
Reliable streamflow forecasting is a determining factor for water resource planning and flood control. To better understand the strengths and weaknesses of newly proposed methods in streamflow forecasting and facilitate comparisons of different research results, we test a simple, universal, and efficient benchmark method, namely, the naïve method, for short-term streamflow prediction. Using the naïve method, we assess the streamflow forecasting performance of the long short-term memory models trained with different objective functions, including mean squared error (MSE), root mean squared error (RMSE), Nash–Sutcliffe efficiency (NSE), Kling–Gupta efficiency (KGE), and mean absolute error (MAE). The experiments over 273 watersheds show that the naïve method attains good forecasting performance (NSE > 0.5) in 88%, 65%, and 52% of watersheds at lead times of 1 day, 2 days, and 3 days, respectively. Through benchmarking by the naïve method, we find that the LSTM models trained with squared-error-based objective functions, i.e., MSE, RMSE, NSE, and KGE, perform poorly in low flow forecasting. This is because they are more influenced by training samples with high flows than by those with low flows during the model training process. For comprehensive short-term streamflow modeling without special demand orientation, we recommend the application of MAE instead of a squared-error-based metric as the objective function. In addition, it is also feasible to perform logarithmic transformation on the streamflow data. This work underscores the critical importance of appropriately selecting the objective functions for model training/calibration, shedding light on how to effectively evaluate the performance of streamflow forecast models.Keywords
Funding Information
- National Natural Science Foundation of China (52079151, 52111540261)
This publication has 49 references indexed in Scilit:
- A Comparative Study on Forecasting of Long-term Daily Streamflow using ANN, ANFIS, BiLSTM and CNN-GRU-LSTMWater Resources Management, 2023
- A Hybrid Model Combining the Cama-Flood Model and Deep Learning Methods for Streamflow PredictionWater Resources Management, 2023
- Monthly River Discharge Forecasting Using Hybrid Models Based on Extreme Gradient Boosting Coupled with Wavelet Theory and Lévy–Jaya Optimization AlgorithmWater Resources Management, 2023
- Performance Improvement of LSTM-based Deep Learning Model for Streamflow Forecasting Using Kalman FilteringWater Resources Management, 2023
- Assessment of the WRF-Hydro uncoupled hydro-meteorological model on flashy watersheds of the Grande Terre tropical island of New Caledonia (South-West Pacific)Journal of Hydrology: Regional Studies, 2022
- A review of hydrologic signatures and their applicationsWIREs Water, 2020
- Toward Forecasting Groundwater Table in Flood Prone Coastal Cities Using Long Short-term Memory and Recurrent Neural NetworksESS Open Archive, 2019
- Monthly runoff forecasting based on LSTM–ALO modelStochastic Environmental Research and Risk Assessment, 2018
- The Xinanjiang model applied in ChinaJournal of Hydrology, 1992
- River flow forecasting through conceptual models part I — A discussion of principlesJournal of Hydrology, 1970