Method for predicting eutrophication in water bodies based on dual experience pool TDDPG and DDPG-dual 3Q learning mode
Open Access
- 20 June 2026
- journal article
- Published by IWA Publishing in Water Reuse
- Vol. 16 (3) , 505-526
Abstract
A three-layer flow diagram of a deep reinforcement learning framework for water eutrophication prediction. It shows multi-factor water quality data input and preprocessing, three core models (DDPG-Double 3Q fusion, improved Double 3Q learning with LSTM and Transformer, and dual experience pool TDDPG), and output evaluation with prediction curves and metrics (MAE 0.012, RMSE 0.018, MAPE 3.5%, TLI 52.4). Water eutrophication prediction remains challenging due to poor long-term feature retention, susceptibility to local optima, and difficulties in balancing smooth and abrupt time-series patterns. To address these issues, this study develops a two-stage reinforcement-learning forecasting framework in which Transformer-based temporal representation, auxiliary replay learning, and error-aware Q-value selection are jointly organized for multivariate eutrophication prediction. In the first stage, the TDDPG model replaces the conventional actor representation in DDPG with a Transformer-based temporal feature extractor and uses an auxiliary replay buffer to reduce the effect of strongly correlated sequential samples during policy learning. In the second stage, the DDPG-Double 3Q model treats the outputs and errors of several DDPG-based predictors as decision states, allowing the final prediction policy to select and refine candidate predictions under both gradual and abrupt water-quality variations. Experimental validation using multi-factor water quality monitoring data demonstrates that the proposed framework achieves an average improvement of 35% across key evaluation metrics – Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE) compared to baseline models such as ADDPG and RDPG. The results indicate that the framework improves prediction accuracy and training stability in the tested dataset, suggesting that reinforcement learning can provide a useful sequential decision-making formulation for multivariate eutrophication forecasting.Keywords
This publication has 25 references indexed in Scilit:
- A review of recent hybridized machine learning methodologies for time series forecasting on water-related variablesJournal of Hydrology, 2025
- Machine learning-based evolution of water quality prediction model: An integrated robust framework for comparative application on periodic return and jitter dataEnvironmental Pollution, 2025
- Implementation of enhanced biological phosphorus recovery for phosphorus mining from eutrophic marine sediments: The optimization of parameters and exploration of microbial responsesChemical Engineering Journal, 2024
- Diffusion probabilistic model for bike-sharing demand recovery with factual knowledge fusionNeural Networks, 2024
- Integrating temporal decomposition and data-driven approaches for predicting coastal harmful algal bloomsJournal of Environmental Management, 2024
- A survey on interpretable reinforcement learningMachine Learning, 2024
- Using monitoring and mechanistic modeling to improve understanding of eutrophication in a shallow New England estuaryJournal of Environmental Management, 2024
- Modelling of nearshore microbial water quality at confluence of a local tributary in Lake St. ClairJournal of Great Lakes Research, 2022
- Early warning of cyanobacteria blooms outbreak based on stoichiometric analysis and catastrophe theory modelJournal of Mathematical Chemistry, 2019
- A novel water quality mechanism modeling and eutrophication risk assessment method of lakes and reservoirsNonlinear Dynamics, 2019