Derong Liu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Derong Liu has a total of 160 co-authors. Here are the top co-authors by number of shared publications:
- Bo Zhao
School of Systems Science, Beijing Normal University, Beijing, China
32 shared publications - Ding Wang
Beijing University of Technology
26 shared publications - Biao Luo
Central South University
21 shared publications Ding Wang
Chinese Academy of Sciences
16 shared publicationsHongliang Li
Tencent Inc
15 shared publications- Qinglai Wei
Chinese Academy of Sciences
15 shared publications
Latest Publications
- Open Access
- 1 October 2026
- Published by KEAI Publishing in Journal of Automation and Intelligence
Probabilistic forecasting for wind power generation via enhanced conditional variational autoencoder with normalization flow
- 1 October 2026
- Published by KEAI Publishing in Journal of Automation and Intelligence
Open AccessProbabilistic forecasting for wind power generation via enhanced conditional variational autoencoder with normalization flow
- 23 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. PP  (21682267) , 1-14
Recent Advances on Off-Policy Reinforcement Learning for Optimization Control
Reinforcement learning (RL), a key artificial intelligence technique, has been widely studied and applied over the past two decades to solve various optimization control problems. Generally speaking, there are two basic frameworks for RL-based control design, i.e., on-policy and off-policy RL (OffP-RL). The essential distinction between the two frameworks lies in whether the policy used to generate training data is the behavior policy or the target policy. In on-policy RL-based control methods, the data used for evaluating the target control policy at each iteration must be collected from the system under the target policy itself. In contrast, in OffP-RL methods, the system data is generated by other behavior control policies. It addresses the inadequate exploration problem in on-policy RL methods, making OffP-RL methods more practical and easier to implement. In this article, the recent advances in OffP-RL-based control methods are classified into three categories based on the number of controllers/players involved, i.e., single-/two-/multiplayer. For the single-player case, it is an optimal control problem, which aims to use OffP-RL to learn the optimal control policy, which minimizes the performance index. In the two-player case, most works focus on the $H_{\infty } $ control problem and the two-player zero-sum game, using learning to find the Nash equilibrium. In the multiplayer case, a multiplayer game involves a single system with multiple control inputs, while a multiagent system consists of multiple systems with independent control inputs. Finally, related applications of OffP-RL-based control and future work are analyzed.
- 23 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. PP  (21682267) , 1-14
Recent Advances on Off-Policy Reinforcement Learning for Optimization Control
Reinforcement learning (RL), a key artificial intelligence technique, has been widely studied and applied over the past two decades to solve various optimization control problems. Generally speaking, there are two basic frameworks for RL-based control design, i.e., on-policy and off-policy RL (OffP-RL). The essential distinction between the two frameworks lies in whether the policy used to generate training data is the behavior policy or the target policy. In on-policy RL-based control methods, the data used for evaluating the target control policy at each iteration must be collected from the system under the target policy itself. In contrast, in OffP-RL methods, the system data is generated by other behavior control policies. It addresses the inadequate exploration problem in on-policy RL methods, making OffP-RL methods more practical and easier to implement. In this article, the recent advances in OffP-RL-based control methods are classified into three categories based on the number of controllers/players involved, i.e., single-/two-/multiplayer. For the single-player case, it is an optimal control problem, which aims to use OffP-RL to learn the optimal control policy, which minimizes the performance index. In the two-player case, most works focus on the $H_{\infty } $ control problem and the two-player zero-sum game, using learning to find the Nash equilibrium. In the multiplayer case, a multiplayer game involves a single system with multiple control inputs, while a multiagent system consists of multiple systems with independent control inputs. Finally, related applications of OffP-RL-based control and future work are analyzed.
- Top Cited Papers
- 19 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 56  (6) , 3733-3744
Dynamic Event-Triggered Control for Human–Machine Cooperative Systems Based on Dynamic Authority Allocation
This article addresses the challenging problem of constrained optimal control for human–machine systems subject to external disturbances and the bounded rationality of the human operator. To this end, a novel game-theoretic framework is proposed. Unlike monolithic game formulations, the framework uniquely disaggregates the control problem by transforming it into a multifaceted game via logarithmic barrier functions (BFs): it models human–machine cooperation as a positive-sum game oriented toward shared objectives, and disturbance rejection as a zero-sum game tailored for robustness enhancement. To capture the nonideal human decision-making, we integrate the level- $k$ reasoning framework to model the operator’s bounded cognitive dynamics. The corresponding coupled Hamilton–Jacobi–Isaacs (HJI) equations for this human–machine game are derived, and critically, a rigorous proof of global asymptotic stability (GAS) for the transformed system is provided, establishing a solid theoretical foundation. For online implementation without requiring prior knowledge of the system dynamics, we develop a resource-efficient learning architecture based on the adaptive dynamic programming (ADP) and a novel dynamic event-triggered mechanism (DETM). A key feature of this architecture is a fuzzy logic-based module for dynamic authority allocation, which adaptively adjusts control sharing in real time. Rigorous analysis demonstrates that all signals in the closed-loop system are uniformly ultimately bounded and that Zeno behavior is precluded. Simulation results are presented to validate the effectiveness and superiority of the proposed control strategy.
- 19 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 56  (6) , 3733-3744
Top Cited PapersDynamic Event-Triggered Control for Human–Machine Cooperative Systems Based on Dynamic Authority Allocation
This article addresses the challenging problem of constrained optimal control for human–machine systems subject to external disturbances and the bounded rationality of the human operator. To this end, a novel game-theoretic framework is proposed. Unlike monolithic game formulations, the framework uniquely disaggregates the control problem by transforming it into a multifaceted game via logarithmic barrier functions (BFs): it models human–machine cooperation as a positive-sum game oriented toward shared objectives, and disturbance rejection as a zero-sum game tailored for robustness enhancement. To capture the nonideal human decision-making, we integrate the level- $k$ reasoning framework to model the operator’s bounded cognitive dynamics. The corresponding coupled Hamilton–Jacobi–Isaacs (HJI) equations for this human–machine game are derived, and critically, a rigorous proof of global asymptotic stability (GAS) for the transformed system is provided, establishing a solid theoretical foundation. For online implementation without requiring prior knowledge of the system dynamics, we develop a resource-efficient learning architecture based on the adaptive dynamic programming (ADP) and a novel dynamic event-triggered mechanism (DETM). A key feature of this architecture is a fuzzy logic-based module for dynamic authority allocation, which adaptively adjusts control sharing in real time. Rigorous analysis demonstrates that all signals in the closed-loop system are uniformly ultimately bounded and that Zeno behavior is precluded. Simulation results are presented to validate the effectiveness and superiority of the proposed control strategy.
- 11 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 10  (2) , 1298-1310
CR-LSO: Convex Neural Architecture Optimization in the Latent Space of Graph Variational Autoencoder With Input Convex Neural Networks
In neural architecture search (NAS) methods based on latent space optimization (LSO), a deep generative model is trained to embed discrete neural architectures into a continuous latent space. In this case, different optimization algorithms that operate in the continuous space can be implemented to search neural architectures. However, the optimization of latent variables is challenging for gradient-based LSO since the mapping from the latent space to the architecture performance is generally non-convex. To tackle this problem, the present paper develops a convexity regularized latent space optimization (CR-LSO) method, which aims to regularize the learning process of latent space in order to obtain a convex architecture performance mapping. Specifically, CR-LSO trains a graph variational autoencoder (G-VAE) to learn the continuous representations of discrete architectures. Simultaneously, the learning process of latent space is regularized by the guaranteed convexity of input convex neural networks (ICNNs). In this way, the G-VAE is forced to learn a convex mapping from the architecture representation to the architecture performance. Hereafter, the CR-LSO approximates the performance mapping using the ICNN and leverages the estimated gradient to optimize neural architecture representations. Experimental results on three popular NAS benchmarks show that CR-LSO achieves competitive evaluation results in terms of both computational complexity and architecture performance.
- 11 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 10  (2) , 1298-1310
CR-LSO: Convex Neural Architecture Optimization in the Latent Space of Graph Variational Autoencoder With Input Convex Neural Networks
In neural architecture search (NAS) methods based on latent space optimization (LSO), a deep generative model is trained to embed discrete neural architectures into a continuous latent space. In this case, different optimization algorithms that operate in the continuous space can be implemented to search neural architectures. However, the optimization of latent variables is challenging for gradient-based LSO since the mapping from the latent space to the architecture performance is generally non-convex. To tackle this problem, the present paper develops a convexity regularized latent space optimization (CR-LSO) method, which aims to regularize the learning process of latent space in order to obtain a convex architecture performance mapping. Specifically, CR-LSO trains a graph variational autoencoder (G-VAE) to learn the continuous representations of discrete architectures. Simultaneously, the learning process of latent space is regularized by the guaranteed convexity of input convex neural networks (ICNNs). In this way, the G-VAE is forced to learn a convex mapping from the architecture representation to the architecture performance. Hereafter, the CR-LSO approximates the performance mapping using the ICNN and leverages the estimated gradient to optimize neural architecture representations. Experimental results on three popular NAS benchmarks show that CR-LSO achieves competitive evaluation results in terms of both computational complexity and architecture performance.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Reliability
- Vol. 75, 3250-3263
Reliable Task Allocation for Multirobot System Using Safe Deep Reinforcement Learning
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Reliability
- Vol. 75, 3250-3263
Reliable Task Allocation for Multirobot System Using Safe Deep Reinforcement Learning
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. PP, 1-14
AHRL-PM: Asynchronous Hierarchical Reinforcement Learning Framework for Enhanced Portfolio Management
Effective portfolio management (PM) is a cornerstone of financial strategy, yet it is often challenged by the uncertainties and the high dimensionality of market data. Traditional PM techniques, whether model-based or model-free, frequently fail to address these complexities, resulting in suboptimal asset allocation. This study introduces an asynchronous hierarchical reinforcement learning (AHRL-PM) framework aimed at enhancing active PM. The proposed AHRL-PM approach employs a two-tiered agent system to navigate market complexities. The first-layer agent selects top-performing stocks monthly based on alpha factors, while the second-layer agent adjusts asset weights using market fundamentals and technical indicators, dynamically rebalancing the portfolio in response to market shifts on a daily basis. We rigorously evaluated our AHRL-PM framework through extensive experiments in six globally representative markets. Performance was assessed using two key metrics: annualized return (AR) and Sharpe ratio (SR). Our model consistently outperformed traditional benchmarks on both metrics across all the markets, demonstrating robustness and versatility in PM. Furthermore, ablation studies comparing the AHRL-PM model to a synchronous model and single-layer reinforcement learning (RL) approaches validated its superior performance in terms of profitability and risk-adjusted returns, while also highlighting significant reductions in training time and appropriate portfolio weight adjustments to respond to market dynamics and uncertainties, underscoring the model's efficiency and practical applicability.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. PP, 1-14
AHRL-PM: Asynchronous Hierarchical Reinforcement Learning Framework for Enhanced Portfolio Management
Effective portfolio management (PM) is a cornerstone of financial strategy, yet it is often challenged by the uncertainties and the high dimensionality of market data. Traditional PM techniques, whether model-based or model-free, frequently fail to address these complexities, resulting in suboptimal asset allocation. This study introduces an asynchronous hierarchical reinforcement learning (AHRL-PM) framework aimed at enhancing active PM. The proposed AHRL-PM approach employs a two-tiered agent system to navigate market complexities. The first-layer agent selects top-performing stocks monthly based on alpha factors, while the second-layer agent adjusts asset weights using market fundamentals and technical indicators, dynamically rebalancing the portfolio in response to market shifts on a daily basis. We rigorously evaluated our AHRL-PM framework through extensive experiments in six globally representative markets. Performance was assessed using two key metrics: annualized return (AR) and Sharpe ratio (SR). Our model consistently outperformed traditional benchmarks on both metrics across all the markets, demonstrating robustness and versatility in PM. Furthermore, ablation studies comparing the AHRL-PM model to a synchronous model and single-layer reinforcement learning (RL) approaches validated its superior performance in terms of profitability and risk-adjusted returns, while also highlighting significant reductions in training time and appropriate portfolio weight adjustments to respond to market dynamics and uncertainties, underscoring the model's efficiency and practical applicability.
- Open Access
- 1 December 2025
- Published by KEAI Publishing in Journal of Automation and Intelligence
- Vol. 4  (4) , 254-265
CLAD: Criterion learner and attention distillation for automated CNN prunning
- 1 December 2025
- Published by KEAI Publishing in Journal of Automation and Intelligence
- Vol. 4  (4) , 254-265
Open AccessCLAD: Criterion learner and attention distillation for automated CNN prunning
- 22 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 56  (2) , 915-926
Federated Learning Adaptive Dynamic Programming for Massive Multiagent Mean-Field Games-Based Optimal Consensus
Massive multiagent systems typically involve a very large number of interactions and conflicts of interest among agents, which presents a significant challenge for achieving stable and efficient adaptive optimal consensus control in real-time. To fill this gap, this article develops a novel federated learning adaptive dynamic programming (FL-ADP) control scheme to solve the massive multiagent mean-field games (MFGs)-based optimal consensus problem. First, the complex interactions of each individual agent with all other agents can be approximated by an average or collective influence in the context of MFGs. Then, a novel undiscounted performance index function involving the mean-field coupling term, the tracking errors and their derivatives is proposed to circumvent the potential impact of the improper discount factor selection and achieve better control performance. By designing the critic–mass neural network structure, the coupled Hamilton–Jacobi–Bellman and Fokker–Planck–Kolmogorov equations are solved to derive the approximate optimal control policy and quantify the probability density function of the collective behavior simultaneously. Additionally, to comply with the required convergence condition of the MFG, a novel event-triggered federated learning mechanism is formulated, which achieves a balance between communication resource consumption and the guarantee of algorithm convergence. On the basis of Lyapunov’s direct method, the tracking errors and the weight estimation errors of all agents are guaranteed to be uniformly ultimately bounded. Simulation results of massive multi-uncrewed aerial vehicle systems affirm the rationality and effectiveness of the proposed method.
- 22 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 56  (2) , 915-926
Federated Learning Adaptive Dynamic Programming for Massive Multiagent Mean-Field Games-Based Optimal Consensus
Massive multiagent systems typically involve a very large number of interactions and conflicts of interest among agents, which presents a significant challenge for achieving stable and efficient adaptive optimal consensus control in real-time. To fill this gap, this article develops a novel federated learning adaptive dynamic programming (FL-ADP) control scheme to solve the massive multiagent mean-field games (MFGs)-based optimal consensus problem. First, the complex interactions of each individual agent with all other agents can be approximated by an average or collective influence in the context of MFGs. Then, a novel undiscounted performance index function involving the mean-field coupling term, the tracking errors and their derivatives is proposed to circumvent the potential impact of the improper discount factor selection and achieve better control performance. By designing the critic–mass neural network structure, the coupled Hamilton–Jacobi–Bellman and Fokker–Planck–Kolmogorov equations are solved to derive the approximate optimal control policy and quantify the probability density function of the collective behavior simultaneously. Additionally, to comply with the required convergence condition of the MFG, a novel event-triggered federated learning mechanism is formulated, which achieves a balance between communication resource consumption and the guarantee of algorithm convergence. On the basis of Lyapunov’s direct method, the tracking errors and the weight estimation errors of all agents are guaranteed to be uniformly ultimately bounded. Simulation results of massive multi-uncrewed aerial vehicle systems affirm the rationality and effectiveness of the proposed method.