Mingming Zhao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Mingming Zhao has a total of 38 co-authors. Here are the top co-authors by number of shared publications:
- Ding Wang
Beijing University of Technology
37 shared publications - Junfei Qiao
Beijing University of Technology
29 shared publications - Mingming Ha
School of Intelligence Science and Technology, University of Science and Technology Beijing, Beijing, China
7 shared publications - Lingzhi Hu
School of Information Science and Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing Laboratory of Smart Environmental Protection, and Beijing Institute of Artificial Intelligence, Beijing University of Technology, Beijing, China
4 shared publications - Jin Ren
School of Information Science and Technology, Beijing University of Technology, Beijing, China
3 shared publications - Menghua Li
School of Information Science and Technology, Beijing University of Technology, Beijing, China
3 shared publications
Latest Publications
- 1 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/CAA Journal of Automatica Sinica
- Vol. 13 (5) , 1135-1150
Intelligent Safe Optimal Control Towards Koopman Operator-Driven Nonlinear Systems with Asymmetric State and Input Constraints
For unknown nonlinear systems subject to asymmetric state and input constraints simultaneously, this article establishes a safe value iteration paradigm to learn an optimal control policy in a data-based manner. Initially, the Koopman operator, instead of the black-box neural network, is applied to extract the inherent dynamics of the controlled systems from the measured data, thereby allowing for explicit analysis of the prediction error. To tackle the issue posed by state and input constraints, a crafted control barrier function is seamlessly incorporated into the canonical utility function, which retains the property of positive definiteness for the asymmetric case. Moreover, the value iteration algorithm with regard to the augmented utility function is adopted to attain a safe optimal controller, where the actor and critic networks are leveraged to approximate the control input and associated value function, respectively. The mono-tonicity, safety, and stability of the raised algorithm are further verified rigorously. Via performing three experiments on the linear system, the nonlinear system, and the manipulator plant, comparative results are obtained to substantiate the superiority and efficacy of the developed approach in achieving optimal performance and safe guarantee.
- 1 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/CAA Journal of Automatica Sinica
- Vol. 13 (5) , 1135-1150
Intelligent Safe Optimal Control Towards Koopman Operator-Driven Nonlinear Systems with Asymmetric State and Input Constraints
For unknown nonlinear systems subject to asymmetric state and input constraints simultaneously, this article establishes a safe value iteration paradigm to learn an optimal control policy in a data-based manner. Initially, the Koopman operator, instead of the black-box neural network, is applied to extract the inherent dynamics of the controlled systems from the measured data, thereby allowing for explicit analysis of the prediction error. To tackle the issue posed by state and input constraints, a crafted control barrier function is seamlessly incorporated into the canonical utility function, which retains the property of positive definiteness for the asymmetric case. Moreover, the value iteration algorithm with regard to the augmented utility function is adopted to attain a safe optimal controller, where the actor and critic networks are leveraged to approximate the control input and associated value function, respectively. The mono-tonicity, safety, and stability of the raised algorithm are further verified rigorously. Via performing three experiments on the linear system, the nonlinear system, and the manipulator plant, comparative results are obtained to substantiate the superiority and efficacy of the developed approach in achieving optimal performance and safe guarantee.
- 1 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 22 (7) , 6058-6069
Data-Driven Adaptive Critic Designs for Hybrid Lifelong Learning in Wastewater Treatment Processes
Wastewater treatment yields significant societal benefits in resource recycling, economic development, and public health. Dissolved oxygen (DO) concentration during the wastewater treatment process serves as a critical indicator for assessing effluent quality. Therefore, maintaining DO within an appropriate range is essential. This study proposes a data-driven tracking controller based on an action-dependent heuristic dynamic programming (ADHDP) approach incorporating lifelong learning (LL) to achieve precise DO concentration tracking. First, the LL-ADHDP controller, acting as an auxiliary controller, is integrated with a prior-knowledge-based controller to achieve model-free tracking control. The online LL-ADHDP approach enhances the approximation accuracy of both the critic and action networks. Second, integrating the LL mechanism into these networks mitigates catastrophic forgetting and improves overall robustness. Third, the method is applied to the benchmark simulation model no. 1. Experimental results demonstrate its superior tracking performance. Finally, simulations with diverse reference trajectories confirm the good dynamic performance and effectively reduce the tracking error of the proposed LL-ADHDP method.
- 1 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 22 (7) , 6058-6069
Data-Driven Adaptive Critic Designs for Hybrid Lifelong Learning in Wastewater Treatment Processes
Wastewater treatment yields significant societal benefits in resource recycling, economic development, and public health. Dissolved oxygen (DO) concentration during the wastewater treatment process serves as a critical indicator for assessing effluent quality. Therefore, maintaining DO within an appropriate range is essential. This study proposes a data-driven tracking controller based on an action-dependent heuristic dynamic programming (ADHDP) approach incorporating lifelong learning (LL) to achieve precise DO concentration tracking. First, the LL-ADHDP controller, acting as an auxiliary controller, is integrated with a prior-knowledge-based controller to achieve model-free tracking control. The online LL-ADHDP approach enhances the approximation accuracy of both the critic and action networks. Second, integrating the LL mechanism into these networks mitigates catastrophic forgetting and improves overall robustness. Third, the method is applied to the benchmark simulation model no. 1. Experimental results demonstrate its superior tracking performance. Finally, simulations with diverse reference trajectories confirm the good dynamic performance and effectively reduce the tracking error of the proposed LL-ADHDP method.
- 19 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 23 (15455955) , 5869-5883
Incremental Q -Learning for Data-Driven Adaptive Critic Control of Wastewater Treatment Processes With State Constraints
With the widespread adoption of massive multiple-input multiple-output (MIMO) technology, itThe wastewater treatment process (WWTP) can effectively convert wastewater into reusable water, with dissolved oxygen (DO) concentration playing a crucial role in treatment efficiency. To achieve online control of DO concentration, a data-driven incremental Q-learning (IQL) method with utility reshaping is proposed. The core idea is to integrate the incremental information of control inputs, the norm value of tracking errors, and a control barrier function into the utility function. First, the IQL method designs an online controller by utilizing tracking errors and the incremental information of control inputs, which reduces the approximation pressure on neural networks caused by excessively large control inputs. Second, the IQL leverages the norm value of tracking errors to influence the magnitude of the incremental control inputs, thereby enhancing the adaptive capability and online stability. Third, an adjustable control barrier function is employed to ensure that the tracking errors do not exceed the predefined constraints, thus meeting the specified tracking accuracy requirements. Fourth, a stability criterion related to learning rates is established to demonstrate the convergence of neural network weights. Finally, the superior control performance of the IQL method is validated via simulation results under two distinct weather conditions.
- 19 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 23 (15455955) , 5869-5883
Incremental Q -Learning for Data-Driven Adaptive Critic Control of Wastewater Treatment Processes With State Constraints
With the widespread adoption of massive multiple-input multiple-output (MIMO) technology, itThe wastewater treatment process (WWTP) can effectively convert wastewater into reusable water, with dissolved oxygen (DO) concentration playing a crucial role in treatment efficiency. To achieve online control of DO concentration, a data-driven incremental Q-learning (IQL) method with utility reshaping is proposed. The core idea is to integrate the incremental information of control inputs, the norm value of tracking errors, and a control barrier function into the utility function. First, the IQL method designs an online controller by utilizing tracking errors and the incremental information of control inputs, which reduces the approximation pressure on neural networks caused by excessively large control inputs. Second, the IQL leverages the norm value of tracking errors to influence the magnitude of the incremental control inputs, thereby enhancing the adaptive capability and online stability. Third, an adjustable control barrier function is employed to ensure that the tracking errors do not exceed the predefined constraints, thus meeting the specified tracking accuracy requirements. Fourth, a stability criterion related to learning rates is established to demonstrate the convergence of neural network weights. Finally, the superior control performance of the IQL method is validated via simulation results under two distinct weather conditions.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 23, 17279-17290
Evolving Consensus Control With Iterative Q-Learning for Discrete-Time Multi-Agent Systems
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 23, 17279-17290
Evolving Consensus Control With Iterative Q-Learning for Discrete-Time Multi-Agent Systems
- 29 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 22 (1) , 407-417
Incremental Critic Learning Control With Policy Transfer for Wastewater Treatment Processes
Regarding the control of dissolved oxygen concentration in wastewater treatment plants (WWTPs), the direct control policies in conventional critic learning control algorithms often exhibit high sensitivity to system disturbances, which may potentially causing significant degradation in control performance. To address this limitation, a transferable incremental critic learning control (TICLC) algorithm is developed, reformulating the control policy into an incremental form. Furthermore, the historical operation data of WWTPs is leveraged to enable policy transfer across different control methods, which effectively improves the control performance of the TICLC algorithm during the initial learning phase. In addition, a novel utility function incorporating a soft policy constraint mechanism is designed to enhance the stability of the TICLC algorithm during the online optimization. The stability of the TICLC algorithm is proved by constructing a Lyapunov function. Experimental results derived from the benchmark simulation model no. 1 show that the TICLC algorithm achieves superior control performance for the dissolved oxygen concentration in WWTPs.
- 29 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 22 (1) , 407-417
Incremental Critic Learning Control With Policy Transfer for Wastewater Treatment Processes
Regarding the control of dissolved oxygen concentration in wastewater treatment plants (WWTPs), the direct control policies in conventional critic learning control algorithms often exhibit high sensitivity to system disturbances, which may potentially causing significant degradation in control performance. To address this limitation, a transferable incremental critic learning control (TICLC) algorithm is developed, reformulating the control policy into an incremental form. Furthermore, the historical operation data of WWTPs is leveraged to enable policy transfer across different control methods, which effectively improves the control performance of the TICLC algorithm during the initial learning phase. In addition, a novel utility function incorporating a soft policy constraint mechanism is designed to enhance the stability of the TICLC algorithm during the online optimization. The stability of the TICLC algorithm is proved by constructing a Lyapunov function. Experimental results derived from the benchmark simulation model no. 1 show that the TICLC algorithm achieves superior control performance for the dissolved oxygen concentration in WWTPs.
- 2 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 55 (7) , 3511-3524
Accelerated Value Iteration-Based Safe Q-Learning for Data-Driven Optimal Tracking Control
In this article, an accelerated value iteration-based safe Q-learning (SQL) algorithm is developed to design the tracking controller for unknown nonlinear systems. First, an augmented Q-function, consisting of a quadratic utility function and an adjustable positive-definite control barrier function (CBF), is devised to ensure both the optimality and safety of the tracking controller. The quadratic utility function, associated with optimality, guarantees that the tracking controller can eliminate the ultimate tracking error, regardless of the reference trajectory. The adjustable positive-definite CBF, pertaining to safety, ensures that the tracking error converges faster toward zero while remaining within the safe set at all times. Second, an accelerated iterative learning mechanism, comprising policy evaluation (PE) and policy improvement (PI), is employed to discover the safe optimal tracking control policy. Integrating the difference between two iterative Q-functions into the current PE process can expedite the convergence rate of the SQL algorithm. A policy optimization technique based on Nesterov Momentum method is utilized to accelerate the PI process of the SQL algorithm. When faced with a large amount of offline data, the two-stage accelerated learning effectively reduces computational pressure. Furthermore, convergence of the Q-function sequence and safety of the optimal tracking policy are theoretically analyzed. Finally, by using neural networks and the action-critic structure, two simulation examples are performed to verify the availability of accelerated SQL methods.
- 2 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 55 (7) , 3511-3524
Accelerated Value Iteration-Based Safe Q-Learning for Data-Driven Optimal Tracking Control
In this article, an accelerated value iteration-based safe Q-learning (SQL) algorithm is developed to design the tracking controller for unknown nonlinear systems. First, an augmented Q-function, consisting of a quadratic utility function and an adjustable positive-definite control barrier function (CBF), is devised to ensure both the optimality and safety of the tracking controller. The quadratic utility function, associated with optimality, guarantees that the tracking controller can eliminate the ultimate tracking error, regardless of the reference trajectory. The adjustable positive-definite CBF, pertaining to safety, ensures that the tracking error converges faster toward zero while remaining within the safe set at all times. Second, an accelerated iterative learning mechanism, comprising policy evaluation (PE) and policy improvement (PI), is employed to discover the safe optimal tracking control policy. Integrating the difference between two iterative Q-functions into the current PE process can expedite the convergence rate of the SQL algorithm. A policy optimization technique based on Nesterov Momentum method is utilized to accelerate the PI process of the SQL algorithm. When faced with a large amount of offline data, the two-stage accelerated learning effectively reduces computational pressure. Furthermore, convergence of the Q-function sequence and safety of the optimal tracking policy are theoretically analyzed. Finally, by using neural networks and the action-critic structure, two simulation examples are performed to verify the availability of accelerated SQL methods.
- 8 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 55 (7) , 4587-4599
Iterative Q -Learning Design for Zero-Sum Games With Evolving Policies
This article aims to achieve data-based online evolving control for zero-sum games with unknown dynamics. First of all, the value-iteration-based Q-learning framework is established. Relevant properties of the iterative Q-learning framework are analyzed, including the convergence and monotonicity. Then, the stability property is investigated and the online data is employed for off-policy learning. More importantly, two effective algorithms are designed to achieve online evolving control. In one algorithm, the monotonically nondecreasing Q-learning sequence requires the admissible criterion to guarantee the stability with the simple Q-function initialization. In another algorithm, the monotonically nonincreasing Q-function sequence can ensure the stability without the admissible criterion, but it requires an elaborate initial Q-function. In the end, by including two examples of real physical backgrounds, the excellent performance of online evolving control is exhibited with the given algorithms.
- 8 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 55 (7) , 4587-4599
Iterative Q -Learning Design for Zero-Sum Games With Evolving Policies
This article aims to achieve data-based online evolving control for zero-sum games with unknown dynamics. First of all, the value-iteration-based Q-learning framework is established. Relevant properties of the iterative Q-learning framework are analyzed, including the convergence and monotonicity. Then, the stability property is investigated and the online data is employed for off-policy learning. More importantly, two effective algorithms are designed to achieve online evolving control. In one algorithm, the monotonically nondecreasing Q-learning sequence requires the admissible criterion to guarantee the stability with the simple Q-function initialization. In another algorithm, the monotonically nonincreasing Q-function sequence can ensure the stability without the admissible criterion, but it requires an elaborate initial Q-function. In the end, by including two examples of real physical backgrounds, the excellent performance of online evolving control is exhibited with the given algorithms.
- 20 February 2025
- Published by Springer Nature in Nonlinear Dynamics
- Vol. 113 (13) , 16679-16694
Model-free tracking design for nonlinear zero-sum games with an improved utility function
- 20 February 2025
- Published by Springer Nature in Nonlinear Dynamics
- Vol. 113 (13) , 16679-16694
Model-free tracking design for nonlinear zero-sum games with an improved utility function
- 10 February 2025
- Published by Elsevier in Neural Networks
- Vol. 186, 107249
Neural-network-based accelerated safe Q-learning for optimal control of discrete-time nonlinear systems with state constraints
- 10 February 2025
- Published by Elsevier in Neural Networks
- Vol. 186, 107249
Neural-network-based accelerated safe Q-learning for optimal control of discrete-time nonlinear systems with state constraints