Intelligent Critic Design With Policy Transfer for Wastewater Treatment Processes
- 14 October 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 22796-22806
Abstract
Wastewater treatment plays a meaningful role in environmental protection, water recycling and public health, and its optimal control offers significant economic and social value. However, the wastewater treatment plant is a large-scale nonlinear system, and its own operation is affected by a series of uncontrollable factors such as the flow rate and composition of the incoming wastewater. This makes it extremely challenging for traditional control methods to realize precise control. To address these issues, this paper proposes a novel knowledge-guided policy optimization control method by integrating policy transfer and adaptive dynamic programming. First, an adaptive critic policy transfer framework based on the source domain knowledge selection method is introduced to construct knowledge extraction and data mining structures to enhance the learning efficiency of the target domain agent. Second, a source domain knowledge selection module is proposed to adaptively adjust the knowledge extracted by the target domain agent. Third, an adaptive termination module is designed to determine when the immature policy in the source domain should terminate during the target domain learning process. Finally, the theoretical stability of the method in this paper is demonstrated by designing a reasonable Lyapunov function. The system performance of the wastewater treatment plant under dry, rainy, and stormy weather conditions is evaluated, ultimately demonstrating the superior performance and adaptability of the proposed method.Keywords
Funding Information
- National Natural Science Foundation of China (62222301, 62473012, 62021003)
- National Science and Technology Major Project (2021ZD0112302, 2021ZD0112301)
- Beijing Natural Science Foundation (F251019)
This publication has 37 references indexed in Scilit:
- Data-Driven Tracking Control for Nonaffine Yaw Channel of Helicopter via Off-Policy Reinforcement LearningIEEE Transactions on Aerospace and Electronic Systems, 2025
- Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control ApplicationsIEEE/CAA Journal of Automatica Sinica, 2023
- A novel adaptive dynamic programming based on tracking error for nonlinear discrete-time systemsAutomatica, 2021
- Event-triggered PID control for wastewater treatment plantsJournal of Water Process Engineering, 2020
- Self-Learning Optimal Regulation for Discrete-Time Nonlinear Systems Under Event-Driven FormulationIEEE Transactions on Automatic Control, 2019
- Advances in control technologies for wastewater treatment processes: status, challenges, and perspectivesIEEE/CAA Journal of Automatica Sinica, 2019
- Overcoming Exploration in Reinforcement Learning with Demonstrations2018
- Complete stability analysis of a heuristic approximate dynamic programming control designAutomatica, 2015
- Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learningArtificial Intelligence, 1999
- Q-learningMachine Learning, 1992