Ning Gao
Ning Gao has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Ding Wang
Beijing University of Technology
3 shared publications - Junfei Qiao
Beijing University of Technology
2 shared publications - Derong Liu
School of Artificial Intelligence, Anhui University, Hefei, China
1 shared publication - Fan Zheng
State Key Laboratory of Deep Earth Exploration and Imaging, College of Instrumentation & Electrical Engineering, Jilin University, Changchun 130026, People’s Republic of China
1 shared publication - Frank L. Lewis
UTA Research Institute, The University of Texas at Arlington, Arlington, TX, USA
1 shared publication Jiang Xiaoxue
College of Instrumentation and Electrical Engineering, Jilin University, Changchun, China
1 shared publication
Latest Publications
- 14 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 22796-22806
Intelligent Critic Design With Policy Transfer for Wastewater Treatment Processes
Wastewater treatment plays a meaningful role in environmental protection, water recycling and public health, and its optimal control offers significant economic and social value. However, the wastewater treatment plant is a large-scale nonlinear system, and its own operation is affected by a series of uncontrollable factors such as the flow rate and composition of the incoming wastewater. This makes it extremely challenging for traditional control methods to realize precise control. To address these issues, this paper proposes a novel knowledge-guided policy optimization control method by integrating policy transfer and adaptive dynamic programming. First, an adaptive critic policy transfer framework based on the source domain knowledge selection method is introduced to construct knowledge extraction and data mining structures to enhance the learning efficiency of the target domain agent. Second, a source domain knowledge selection module is proposed to adaptively adjust the knowledge extracted by the target domain agent. Third, an adaptive termination module is designed to determine when the immature policy in the source domain should terminate during the target domain learning process. Finally, the theoretical stability of the method in this paper is demonstrated by designing a reasonable Lyapunov function. The system performance of the wastewater treatment plant under dry, rainy, and stormy weather conditions is evaluated, ultimately demonstrating the superior performance and adaptability of the proposed method.
- 14 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 22796-22806
Intelligent Critic Design With Policy Transfer for Wastewater Treatment Processes
Wastewater treatment plays a meaningful role in environmental protection, water recycling and public health, and its optimal control offers significant economic and social value. However, the wastewater treatment plant is a large-scale nonlinear system, and its own operation is affected by a series of uncontrollable factors such as the flow rate and composition of the incoming wastewater. This makes it extremely challenging for traditional control methods to realize precise control. To address these issues, this paper proposes a novel knowledge-guided policy optimization control method by integrating policy transfer and adaptive dynamic programming. First, an adaptive critic policy transfer framework based on the source domain knowledge selection method is introduced to construct knowledge extraction and data mining structures to enhance the learning efficiency of the target domain agent. Second, a source domain knowledge selection module is proposed to adaptively adjust the knowledge extracted by the target domain agent. Third, an adaptive termination module is designed to determine when the immature policy in the source domain should terminate during the target domain learning process. Finally, the theoretical stability of the method in this paper is demonstrated by designing a reasonable Lyapunov function. The system performance of the wastewater treatment plant under dry, rainy, and stormy weather conditions is evaluated, ultimately demonstrating the superior performance and adaptability of the proposed method.
- Top Cited Papers
- 10 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/CAA Journal of Automatica Sinica
- Vol. 11 (1) , 18-36
Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications
Reinforcement learning (RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming (ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively. Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks, showing how they promote ADP formulation significantly. Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has demonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.
- 10 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/CAA Journal of Automatica Sinica
- Vol. 11 (1) , 18-36
Top Cited PapersRecent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications
Reinforcement learning (RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming (ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively. Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks, showing how they promote ADP formulation significantly. Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has demonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.
- 5 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 54 (8) , 4690-4701
Intelligent-Critic-Based Tracking Control of Discrete-Time Input-Affine Systems and Approximation Error Analysis With Application Verification
In recent years, the application of function approximators, such as neural networks and polynomials, has ushered in a new stage of development in solving optimal control problems. However, considering the existence of approximation errors, the stability of the controlled system cannot be guaranteed. Therefore, in view of the prevalence of approximation errors, we investigate optimal tracking control problems for discrete-time systems. First, a novel value function is introduced into the intelligent critic framework. Second, an implicit method is utilized to demonstrate the boundedness of the iterative value functions with approximation errors. An explicit method is applied to prove the stability of the system with approximation errors. Furthermore, an evolving policy is designed to iteratively tackle the optimal tracking control problem and demonstrate the stability of the system. Finally, the effectiveness of the developed method is verified through numerical as well as practical examples.
- 5 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 54 (8) , 4690-4701
Intelligent-Critic-Based Tracking Control of Discrete-Time Input-Affine Systems and Approximation Error Analysis With Application Verification
In recent years, the application of function approximators, such as neural networks and polynomials, has ushered in a new stage of development in solving optimal control problems. However, considering the existence of approximation errors, the stability of the controlled system cannot be guaranteed. Therefore, in view of the prevalence of approximation errors, we investigate optimal tracking control problems for discrete-time systems. First, a novel value function is introduced into the intelligent critic framework. Second, an implicit method is utilized to demonstrate the boundedness of the iterative value functions with approximation errors. An explicit method is applied to prove the stability of the system with approximation errors. Furthermore, an evolving policy is designed to iteratively tackle the optimal tracking control problem and demonstrate the stability of the system. Finally, the effectiveness of the developed method is verified through numerical as well as practical examples.
- 1 October 2013
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 118-121
A remote monitoring system for non-cable self-positioning seismographs
For the convenience of non-cable self-positioning seismographs monitoring in complex terrain, a remote monitoring system is designed and developed. The command interaction and file interaction strategies are created for connecting between the remote monitoring system and non-cable self-positioning seismographs with wireless communication technology, the applications are published on IOS devices which realize status monitoring, command controlling, data harvesting and waveform displaying of non-cable self-positioning seismographs. Indoor tests and field experiments indicate that the remote monitoring system is reliable, stable and operated simply, it assists users to inspect and shoot troubles, and guarantees the quality and efficiency of field exploration.
- 1 October 2013
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 118-121
A remote monitoring system for non-cable self-positioning seismographs
For the convenience of non-cable self-positioning seismographs monitoring in complex terrain, a remote monitoring system is designed and developed. The command interaction and file interaction strategies are created for connecting between the remote monitoring system and non-cable self-positioning seismographs with wireless communication technology, the applications are published on IOS devices which realize status monitoring, command controlling, data harvesting and waveform displaying of non-cable self-positioning seismographs. Indoor tests and field experiments indicate that the remote monitoring system is reliable, stable and operated simply, it assists users to inspect and shoot troubles, and guarantees the quality and efficiency of field exploration.