Incremental Critic Learning Control With Policy Transfer for Wastewater Treatment Processes
- 29 September 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 22 (1) , 407-417
Abstract
Regarding the control of dissolved oxygen concentration in wastewater treatment plants (WWTPs), the direct control policies in conventional critic learning control algorithms often exhibit high sensitivity to system disturbances, which may potentially causing significant degradation in control performance. To address this limitation, a transferable incremental critic learning control (TICLC) algorithm is developed, reformulating the control policy into an incremental form. Furthermore, the historical operation data of WWTPs is leveraged to enable policy transfer across different control methods, which effectively improves the control performance of the TICLC algorithm during the initial learning phase. In addition, a novel utility function incorporating a soft policy constraint mechanism is designed to enhance the stability of the TICLC algorithm during the online optimization. The stability of the TICLC algorithm is proved by constructing a Lyapunov function. Experimental results derived from the benchmark simulation model no. 1 show that the TICLC algorithm achieves superior control performance for the dissolved oxygen concentration in WWTPs.Keywords
Funding Information
- National Science and Technology Major Project (2021ZD0112302)
- National Natural Science Foundation of China (62222301)
- National Natural Science Foundation of China (62473012)
- National Natural Science Foundation of China (62021003)
- Beijing Natural Science Foundation (F251019)
This publication has 37 references indexed in Scilit:
- General multi-step value iteration for optimal learning controlAutomatica, 2025
- Reinforcement learning control with n-step information for wastewater treatment systemsEngineering Applications of Artificial Intelligence, 2024
- Supplementary heuristic dynamic programming for wastewater treatment process controlExpert Systems with Applications, 2024
- Residual-triggered threshold decision and performance self-healing control for wastewater treatment processInformation Sciences, 2023
- Reward shaping to improve the performance of deep reinforcement learning in perishable inventory managementEuropean Journal of Operational Research, 2022
- Adaptable automation with modular deep reinforcement learning and policy transferEngineering Applications of Artificial Intelligence, 2021
- Event-triggered PID control for wastewater treatment plantsJournal of Water Process Engineering, 2020
- Dissolved Oxygen Control in Biological Wastewater Treatments with Non-Ideal Sensors and ActuatorsIndustrial & Engineering Chemistry Research, 2019
- A boundedness result for the direct heuristic dynamic programmingNeural Networks, 2012
- Dissolved oxygen control of the activated sludge wastewater treatment process using model predictive controlComputers & Chemical Engineering, 2008