Incremental Q -Learning for Data-Driven Adaptive Critic Control of Wastewater Treatment Processes With State Constraints
- 19 February 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 23 (15455955) , 5869-5883
Abstract
With the widespread adoption of massive multiple-input multiple-output (MIMO) technology, itThe wastewater treatment process (WWTP) can effectively convert wastewater into reusable water, with dissolved oxygen (DO) concentration playing a crucial role in treatment efficiency. To achieve online control of DO concentration, a data-driven incremental Q-learning (IQL) method with utility reshaping is proposed. The core idea is to integrate the incremental information of control inputs, the norm value of tracking errors, and a control barrier function into the utility function. First, the IQL method designs an online controller by utilizing tracking errors and the incremental information of control inputs, which reduces the approximation pressure on neural networks caused by excessively large control inputs. Second, the IQL leverages the norm value of tracking errors to influence the magnitude of the incremental control inputs, thereby enhancing the adaptive capability and online stability. Third, an adjustable control barrier function is employed to ensure that the tracking errors do not exceed the predefined constraints, thus meeting the specified tracking accuracy requirements. Fourth, a stability criterion related to learning rates is established to demonstrate the convergence of neural network weights. Finally, the superior control performance of the IQL method is validated via simulation results under two distinct weather conditions.Keywords
Funding Information
- National Natural Science Foundation of China (T2442012)
- National Natural Science Foundation of China (62473012)
- National Natural Science Foundation of China (62222301)
- Beijing Natural Science Foundation (F251019)
- National Science and Technology Major Project (2021ZD0112302)
- National Science and Technology Major Project (2021ZD0112301)
This publication has 41 references indexed in Scilit:
- Off-policy safe reinforcement learning for nonlinear discrete-time systemsNeurocomputing, 2024
- Safe exploration in model-based reinforcement learning using control barrier functionsAutomatica, 2022
- Safe reinforcement learning for affine nonlinear systems with state constraints and input saturation using control barrier functionsNeurocomputing, 2022
- Event-triggered PID control for wastewater treatment plantsJournal of Water Process Engineering, 2020
- Impacts of dissolved oxygen control on different greenhouse gas emission sources in wastewater treatment processJournal of Cleaner Production, 2020
- Safe reinforcement learning: A control barrier function optimization approachInternational Journal of Robust and Nonlinear Control, 2020
- Incremental model-based global dual heuristic programming with explicit analytical calculations applied to flight controlEngineering Applications of Artificial Intelligence, 2020
- Event triggered nonlinear model predictive control for a wastewater treatment plantJournal of Water Process Engineering, 2019
- Fuzzy logic for plant-wide control of biological wastewater treatment process including greenhouse gas emissionsISA Transactions, 2018
- Complete stability analysis of a heuristic approximate dynamic programming control designAutomatica, 2015