Shunyu Wu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Shunyu Wu has a total of 21 co-authors. Here are the top co-authors by number of shared publications:
- Jingcheng Wang
Department of Automation, Key Laboratory of System Control and Information Processing, Shanghai Jiao Tong University, Shanghai, China
9 shared publications - Jiahui Xu
Department of Automation, Key Laboratory of System Control and Information Processing, Shanghai Jiao Tong University, Shanghai, China
7 shared publications - Jun Rao
Department of Automation, Key Laboratory of System Control and Information Processing, Shanghai Jiao Tong University, Shanghai, China
7 shared publications - Yanjiu Zhong
Department of Automation, Key Laboratory of System Control and Information Processing, Shanghai Jiao Tong University, Shanghai, China
6 shared publications - Haotian Xu
School of Control Science and Engineering, Shandong University, Jinan, China
2 shared publications Jingcheng Wang
2 shared publications
Latest Publications
- 2 September 2026
- Published by Springer Nature in Applied Intelligence
- Vol. 56 (14) , 400
Optimal fault feature learning for machinery fault diagnosis by using recurrent attentional reinforcement learning
- 2 September 2026
- Published by Springer Nature in Applied Intelligence
- Vol. 56 (14) , 400
Optimal fault feature learning for machinery fault diagnosis by using recurrent attentional reinforcement learning
- 18 June 2026
- Published by Elsevier in Expert Systems with Applications
- Vol. 331
DRIFT: Coordinating target intent and local geometry for diffusion-based trajectory generation
- 18 June 2026
- Published by Elsevier in Expert Systems with Applications
- Vol. 331
DRIFT: Coordinating target intent and local geometry for diffusion-based trajectory generation
- 27 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 16 (6) , 4796-4807
End-to-End Stochastic Predict-Then-Optimize for Cost-Efficient Water-Energy Resource Scheduling
Compared to other energy-intensive industries, urban water supply systems are unique due to the strong temporal coupling with the power grid, which intensifies grid stress during peak hours. This paper addresses the water-energy nexus by developing a water-energy resource scheduling model for wholeprocess water supply that incorporates electricity cost prediction uncertainties. Meanwhile, to minimize the scheduling cost in an end-to-end approach, we propose the Stochastic Predictthen-Optimize (SPTO), which aligns prediction distribution with the optimality gap by integrating probabilistic forecasting with stochastic optimization. Unlike traditional two-stage approaches, SPTO establishes a closed-loop feedback between forecasting and optimization, reducing cumulative errors and improving realtime adaptability. To enable end-to-end training, we derive the SPTO+, a differentiable convex upper-bound surrogate function based on Lagrangian duality, which provides surrogate gradient computation for end-to-end training. Experiments on real-world systems demonstrate that SPTO significantly reduces operational cost and decision regret, effectively coordinating water operations with dynamic grid conditions to support cost-efficient and gridresponsive scheduling.
- 27 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 16 (6) , 4796-4807
End-to-End Stochastic Predict-Then-Optimize for Cost-Efficient Water-Energy Resource Scheduling
Compared to other energy-intensive industries, urban water supply systems are unique due to the strong temporal coupling with the power grid, which intensifies grid stress during peak hours. This paper addresses the water-energy nexus by developing a water-energy resource scheduling model for wholeprocess water supply that incorporates electricity cost prediction uncertainties. Meanwhile, to minimize the scheduling cost in an end-to-end approach, we propose the Stochastic Predictthen-Optimize (SPTO), which aligns prediction distribution with the optimality gap by integrating probabilistic forecasting with stochastic optimization. Unlike traditional two-stage approaches, SPTO establishes a closed-loop feedback between forecasting and optimization, reducing cumulative errors and improving realtime adaptability. To enable end-to-end training, we derive the SPTO+, a differentiable convex upper-bound surrogate function based on Lagrangian duality, which provides surrogate gradient computation for end-to-end training. Experiments on real-world systems demonstrate that SPTO significantly reduces operational cost and decision regret, effectively coordinating water operations with dynamic grid conditions to support cost-efficient and gridresponsive scheduling.
- 26 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 54 (6) , 3809-3821
Parallel Cross Entropy Policy Gradient Adaptive Dynamic Programming for Optimal Tracking Control of Discrete-Time Nonlinear Systems
Policy gradient adaptive dynamic programming (PGADP) is a recently acclaimed control technique for the optimal control design of nonlinear systems. Nevertheless, it demands a substantial amount of interaction data with the controlled system, which can prove costly or perilous in certain scenarios. This article introduces a parallel cross entropy optimization method-based PGADP (PCEOM-PGADP) algorithm, with the objective of devising an optimal tracking controller for discrete-time nonlinear systems. The tracking problem is transformed into a regulation problem by constructing a tracking error system. Furthermore, the implementation of the proposed algorithm employs an actor–critic structure, where the actor network represents the control policy and the critic network assesses its performance. Through the iterative interaction, the optimal policy is ultimately derived. The approach also leverages the parallel cross entropy optimization method (PCEOM) to acquire a reasonable initial control policy for PGADP, thereby accelerating the efficiency of the learning process. Convergence analysis of the algorithm is conducted by demonstrating that the generated $Q$ function constitutes a monotonically nonincreasing sequence. Finally, the effectiveness of the proposed PCEOM-PGADP algorithm is verified through simulation on a complex automated driving tracking system.
- 26 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 54 (6) , 3809-3821
Parallel Cross Entropy Policy Gradient Adaptive Dynamic Programming for Optimal Tracking Control of Discrete-Time Nonlinear Systems
Policy gradient adaptive dynamic programming (PGADP) is a recently acclaimed control technique for the optimal control design of nonlinear systems. Nevertheless, it demands a substantial amount of interaction data with the controlled system, which can prove costly or perilous in certain scenarios. This article introduces a parallel cross entropy optimization method-based PGADP (PCEOM-PGADP) algorithm, with the objective of devising an optimal tracking controller for discrete-time nonlinear systems. The tracking problem is transformed into a regulation problem by constructing a tracking error system. Furthermore, the implementation of the proposed algorithm employs an actor–critic structure, where the actor network represents the control policy and the critic network assesses its performance. Through the iterative interaction, the optimal policy is ultimately derived. The approach also leverages the parallel cross entropy optimization method (PCEOM) to acquire a reasonable initial control policy for PGADP, thereby accelerating the efficiency of the learning process. Convergence analysis of the algorithm is conducted by demonstrating that the generated $Q$ function constitutes a monotonically nonincreasing sequence. Finally, the effectiveness of the proposed PCEOM-PGADP algorithm is verified through simulation on a complex automated driving tracking system.
- 12 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 54 (4) , 2368-2378
Reinforcement Learning Controller Design for Discrete-Time-Constrained Nonlinear Systems With Weight Initialization Method
Extensive research has been dedicated to reinforcement learning (RL) for acquiring proficient optimal controllers through interactions with the environment. However, real-world demands, including enhanced safety performance, introduce considerable challenges to the present design of optimal controllers rooted in RL algorithms. A novel approach is introduced in this article for designing RL-based optimal controllers, employing a control barrier function (CBF) alongside a nonquadratic loss function related to the control signal. The aim is to enable the agent to learn the optimal controller in a secure and efficient manner. To tackle the instability issue in neural network training inherent to traditional RL-based controller design processes, the nonlinear model predictive control (NMPC) technique is employed for initializing the controller network’s weights. A formal demonstration of the method’s optimality is presented. Numerical simulations validate the proposed approach, illustrating its capacity to effectively learn the optimal controller while adhering to the input and state constraints of the system.
- 12 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 54 (4) , 2368-2378
Reinforcement Learning Controller Design for Discrete-Time-Constrained Nonlinear Systems With Weight Initialization Method
Extensive research has been dedicated to reinforcement learning (RL) for acquiring proficient optimal controllers through interactions with the environment. However, real-world demands, including enhanced safety performance, introduce considerable challenges to the present design of optimal controllers rooted in RL algorithms. A novel approach is introduced in this article for designing RL-based optimal controllers, employing a control barrier function (CBF) alongside a nonquadratic loss function related to the control signal. The aim is to enable the agent to learn the optimal controller in a secure and efficient manner. To tackle the instability issue in neural network training inherent to traditional RL-based controller design processes, the nonlinear model predictive control (NMPC) technique is employed for initializing the controller network’s weights. A formal demonstration of the method’s optimality is presented. Numerical simulations validate the proposed approach, illustrating its capacity to effectively learn the optimal controller while adhering to the input and state constraints of the system.
- 13 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2258-2263
Adaptive Optimal Control of Nonlinear Systems with Multiple Time-Scale Eligibility Traces
Adaptive dynamic programming (ADP) is one of the main methods to solve the optimal control problem of nonlinear systems. Eligibility traces are utilized in recent years to reduce the computing burden of the value function, but the existing fixed eligibility trace is difficult to ensure stable convergence especially when facing environmental changes and complex neural network structures. To solve the above issues, a novel off-policy algorithm, $T$ -HDP $(\lambda)$ with Multiple Time-scale Eligibility Traces (MET), is proposed. By utilizing MET, the new algorithm can adaptively accumulate gradients and include more gradient information, which guides the control faster in the optimal direction. $T$ -step Truncated $\lambda$ -returns are utilized to solve the infinite-horizon optimal control problems, and a new importance sampling ratio is designed to correct the value function. Furthermore, the convergence and boundedness of the algorithm are proved. Based on the actor-critic network architecture, the optimal value function and policy are well approximated. Finally, compared with the original algorithm by a simulation example, the proposed algorithm has a faster convergence speed and lower variance.- 13 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2258-2263
Adaptive Optimal Control of Nonlinear Systems with Multiple Time-Scale Eligibility Traces
Adaptive dynamic programming (ADP) is one of the main methods to solve the optimal control problem of nonlinear systems. Eligibility traces are utilized in recent years to reduce the computing burden of the value function, but the existing fixed eligibility trace is difficult to ensure stable convergence especially when facing environmental changes and complex neural network structures. To solve the above issues, a novel off-policy algorithm, $T$ -HDP $(\lambda)$ with Multiple Time-scale Eligibility Traces (MET), is proposed. By utilizing MET, the new algorithm can adaptively accumulate gradients and include more gradient information, which guides the control faster in the optimal direction. $T$ -step Truncated $\lambda$ -returns are utilized to solve the infinite-horizon optimal control problems, and a new importance sampling ratio is designed to correct the value function. Furthermore, the convergence and boundedness of the algorithm are proved. Based on the actor-critic network architecture, the optimal value function and policy are well approximated. Finally, compared with the original algorithm by a simulation example, the proposed algorithm has a faster convergence speed and lower variance.- 16 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 21 (3) , 4585-4598
A Novel Series-Concatenation Hybrid Prediction Model of Energy Consumption in Hot Strip Roughing Process With Multi-Step Rolling
The steel industry has received serious attention under the background of carbon neutralization and carbon peaking. However, the traditional end-to-end energy consumption (EC) prediction method does not consider the effects of multi-step rolling, multi-time series, and error accumulation. To this end, a series-concatenation model based on a two-stage hybrid network is proposed to achieve EC high-precision prediction in multi-step continuous rolling. Specifically. First, this paper analyzed the mechanism between rolling EC and multiple variables. Second, a two-stage hybrid network with a deep neural network block, convolutional neural network block, long short-term memory block, and SoftBoost block (DCLS-Net) is established for EC prediction of single-step rolling. SoftBoost block is a double-layer structure method proposed in this paper for multi-block precision improvement based on two kinds of boosting strategies. Last, according to the error mechanism and multi-time series characteristics in the multi-step rolling, a series-concatenation EC prediction model is designed to suppress errors and achieve high-precision prediction. The experimental results show that the SoftBoost can effectively improve prediction performance. And the prediction precision of the two-stage DCLS-Net for single-step rolling is improved by 9.43% on average compared with the end-to-end machine learning algorithm. Furthermore, the precision of the series-concatenation model for multi-step rolling is improved by 4.96% compared with the traditional series model, which can satisfy the requirements of high accuracy and positive error in strip rolling production. Note to Practitioners —The inspiration for this paper mainly comes from the multi-objective process planning problem in the hot multi-step continuous roughing process, especially the energy consumption target. This method is also applicable to other variable prediction problems in the continuous multi-step process industry. The common method of energy consumption prediction is end-to-end machine learning. However, the characteristics of multi-step rolling and multi-time series, as well as the influence of intermediate process variables on the results, are not considered, which makes the precision and reliability of prediction unable to satisfy the requirements of industrial production. In this paper, the series-concatenation model structure is adopted to realize the high-precision energy consumption prediction and error suppression of multi-step rolling of strip steel. Such research is helpful for engineers to understand the energy consumption mechanism of strip rolling. It provides guidance for multi-objective roughing process planning. In future research, we will study the influence of more process variables on energy consumption.
- 16 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 21 (3) , 4585-4598
A Novel Series-Concatenation Hybrid Prediction Model of Energy Consumption in Hot Strip Roughing Process With Multi-Step Rolling
The steel industry has received serious attention under the background of carbon neutralization and carbon peaking. However, the traditional end-to-end energy consumption (EC) prediction method does not consider the effects of multi-step rolling, multi-time series, and error accumulation. To this end, a series-concatenation model based on a two-stage hybrid network is proposed to achieve EC high-precision prediction in multi-step continuous rolling. Specifically. First, this paper analyzed the mechanism between rolling EC and multiple variables. Second, a two-stage hybrid network with a deep neural network block, convolutional neural network block, long short-term memory block, and SoftBoost block (DCLS-Net) is established for EC prediction of single-step rolling. SoftBoost block is a double-layer structure method proposed in this paper for multi-block precision improvement based on two kinds of boosting strategies. Last, according to the error mechanism and multi-time series characteristics in the multi-step rolling, a series-concatenation EC prediction model is designed to suppress errors and achieve high-precision prediction. The experimental results show that the SoftBoost can effectively improve prediction performance. And the prediction precision of the two-stage DCLS-Net for single-step rolling is improved by 9.43% on average compared with the end-to-end machine learning algorithm. Furthermore, the precision of the series-concatenation model for multi-step rolling is improved by 4.96% compared with the traditional series model, which can satisfy the requirements of high accuracy and positive error in strip rolling production. Note to Practitioners —The inspiration for this paper mainly comes from the multi-objective process planning problem in the hot multi-step continuous roughing process, especially the energy consumption target. This method is also applicable to other variable prediction problems in the continuous multi-step process industry. The common method of energy consumption prediction is end-to-end machine learning. However, the characteristics of multi-step rolling and multi-time series, as well as the influence of intermediate process variables on the results, are not considered, which makes the precision and reliability of prediction unable to satisfy the requirements of industrial production. In this paper, the series-concatenation model structure is adopted to realize the high-precision energy consumption prediction and error suppression of multi-step rolling of strip steel. Such research is helpful for engineers to understand the energy consumption mechanism of strip rolling. It provides guidance for multi-objective roughing process planning. In future research, we will study the influence of more process variables on energy consumption.
- 10 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 54 (3) , 1489-1501
Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems With Trajectory-Based Initial Control Policy
The policy gradient adaptive dynamic programming (PGADP) technique has gained recognition as an effective approach for optimizing the performance of nonlinear systems. Nonetheless, existing PGADP algorithms often demand a substantial volume of expensive or potentially risky interaction data with the system. Moreover, the utilization of neural networks in these algorithms can result in suboptimal learning efficiency and unstable training procedures. To address these challenges, a novel algorithm, referred to as OptNet-PGADP, has been introduced. This algorithm integrates an initially tailored control policy based on OptNet to tackle the optimization of control problems in discrete-time nonlinear systems. The OptNet-PGADP algorithm operates through a two-step process. Initially, the input–output trajectory of the system is computed using the nonlinear model predictive control (NMPC) method. Subsequently, an initial admissible control policy is acquired through OptNet. This policy is iteratively enhanced using the PGADP algorithm to attain the optimal controller. The resulting closed-loop control policy can be readily deployed in real-time applications. The implementation of the algorithm employs OptNet for the actor network and integrates an experience replay mechanism to bolster the controller’s learning efficiency. Furthermore, a convergence and optimality analysis of the algorithm is included. Simulation and experimental results conducted on two nonlinear systems conclusively demonstrate that the approach outperforms traditional PGADP and NMPC algorithms. These findings underscore the efficacy of OptNet-PGADP in mitigating the constraints of current methods and achieving superior control performance for nonlinear systems.
- 10 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 54 (3) , 1489-1501
Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems With Trajectory-Based Initial Control Policy
The policy gradient adaptive dynamic programming (PGADP) technique has gained recognition as an effective approach for optimizing the performance of nonlinear systems. Nonetheless, existing PGADP algorithms often demand a substantial volume of expensive or potentially risky interaction data with the system. Moreover, the utilization of neural networks in these algorithms can result in suboptimal learning efficiency and unstable training procedures. To address these challenges, a novel algorithm, referred to as OptNet-PGADP, has been introduced. This algorithm integrates an initially tailored control policy based on OptNet to tackle the optimization of control problems in discrete-time nonlinear systems. The OptNet-PGADP algorithm operates through a two-step process. Initially, the input–output trajectory of the system is computed using the nonlinear model predictive control (NMPC) method. Subsequently, an initial admissible control policy is acquired through OptNet. This policy is iteratively enhanced using the PGADP algorithm to attain the optimal controller. The resulting closed-loop control policy can be readily deployed in real-time applications. The implementation of the algorithm employs OptNet for the actor network and integrates an experience replay mechanism to bolster the controller’s learning efficiency. Furthermore, a convergence and optimality analysis of the algorithm is included. Simulation and experimental results conducted on two nonlinear systems conclusively demonstrate that the approach outperforms traditional PGADP and NMPC algorithms. These findings underscore the efficacy of OptNet-PGADP in mitigating the constraints of current methods and achieving superior control performance for nonlinear systems.