Wenjun Zou
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Wenjun Zou has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Shengbo Eben Li
Tsinghua University
6 shared publications - Jingliang Duan
School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China
4 shared publications - Yuming Yin
School of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
4 shared publications - Jianyu Chen
Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China
2 shared publications - Yinuo Wang
School of Vehicle and Mobility, Tsinghua University, Beijing, China
2 shared publications - Yujie Yang
School of Vehicle and Mobility, Tsinghua University, Beijing, China
2 shared publications
Latest Publications
- 15 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. PP (10834435) , 1-11
DIK: A Diffusion Inverse Kinematics Method of Robot Manipulators
Inverse kinematics (IK) determines the joint configurations needed to achieve a specific end pose of a kinematic chain. IK solutions for complex robot manipulators are typically nonunique, allowing multiple joint configurations to correspond to the same end pose. However, traditional IK methods generally yield only a single solution. To overcome this limitation, this study introduces the diffusion inverse kinematics (DIK) method, which can provide multiple IK solutions for the same end pose. This capability allows for the selection of the optimal configuration based on criteria, such as accuracy, proximity to the current position, or adherence to specific constraints. DIK utilizes a diffusion generation process akin to the sampling process of the diffusion model to generate these multimodal solutions, and it employs a self-supervised learning framework to align the training data with real-world applications. This framework updates the parameters of the diffusion generation process by minimizing the difference between the actual poses and the target poses. For robot manipulators that operate under specific constraints, DIK integrates these constraints into the IK solution process, and addresses them using exterior point methods. Experimental results affirm that DIK not only generates multimodal IK solutions but also achieves high accuracy and zero-collision outcomes.
- 15 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. PP (10834435) , 1-11
DIK: A Diffusion Inverse Kinematics Method of Robot Manipulators
Inverse kinematics (IK) determines the joint configurations needed to achieve a specific end pose of a kinematic chain. IK solutions for complex robot manipulators are typically nonunique, allowing multiple joint configurations to correspond to the same end pose. However, traditional IK methods generally yield only a single solution. To overcome this limitation, this study introduces the diffusion inverse kinematics (DIK) method, which can provide multiple IK solutions for the same end pose. This capability allows for the selection of the optimal configuration based on criteria, such as accuracy, proximity to the current position, or adherence to specific constraints. DIK utilizes a diffusion generation process akin to the sampling process of the diffusion model to generate these multimodal solutions, and it employs a self-supervised learning framework to align the training data with real-world applications. This framework updates the parameters of the diffusion generation process by minimizing the difference between the actual poses and the target poses. For robot manipulators that operate under specific constraints, DIK integrates these constraints into the IK solution process, and addresses them using exterior point methods. Experimental results affirm that DIK not only generates multimodal IK solutions but also achieves high accuracy and zero-collision outcomes.
- 12 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 24105-24119
Multimodal Reinforcement Learning With Score-Based Policy
Learning multimodal policies is crucial for enhancing exploration in online reinforcement learning (RL), especially in tasks with continuous action spaces and non-convex reward landscapes. While recent diffusion policies show promise, they often suffer from low computational efficiency in online settings. A more training-efficient paradigm involves modeling the policy as a Boltzmann distribution and guiding the action sampling directly with the gradient of the Q-value with respect to the action (proportional to the score function of the policy), such as via Langevin dynamics. However, analysis in this paper reveals that this gradient-guided approach suffers from two critical challenges: sampling instability caused by the widely varying magnitude of action gradients; and mode imbalance, where the sampling process inaccurately represents the weights of different high-value action modes. To address these challenges, this paper introduces three targeted techniques: score normalization and reshaping to stabilize the sampling process, and value-based resampling to correct mode imbalance. These techniques are then integrated into an actor-critic framework, resulting in the Score-Enhanced Actor-Critic (SEAC) algorithm. Simulation and real-world experiments demonstrate that SEAC not only effectively learns multimodal behaviors but also achieves state-of-the-art performance and high computational efficiency compared to prior multimodal RL methods. The code of this paper is available at https://github.com/THUzouwenjun/SEAC.
- 12 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 24105-24119
Multimodal Reinforcement Learning With Score-Based Policy
Learning multimodal policies is crucial for enhancing exploration in online reinforcement learning (RL), especially in tasks with continuous action spaces and non-convex reward landscapes. While recent diffusion policies show promise, they often suffer from low computational efficiency in online settings. A more training-efficient paradigm involves modeling the policy as a Boltzmann distribution and guiding the action sampling directly with the gradient of the Q-value with respect to the action (proportional to the score function of the policy), such as via Langevin dynamics. However, analysis in this paper reveals that this gradient-guided approach suffers from two critical challenges: sampling instability caused by the widely varying magnitude of action gradients; and mode imbalance, where the sampling process inaccurately represents the weights of different high-value action modes. To address these challenges, this paper introduces three targeted techniques: score normalization and reshaping to stabilize the sampling process, and value-based resampling to correct mode imbalance. These techniques are then integrated into an actor-critic framework, resulting in the Score-Enhanced Actor-Critic (SEAC) algorithm. Simulation and real-world experiments demonstrate that SEAC not only effectively learns multimodal behaviors but also achieves state-of-the-art performance and high computational efficiency compared to prior multimodal RL methods. The code of this paper is available at https://github.com/THUzouwenjun/SEAC.
- 24 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 9 (11) , 7059-7069
Fast Online Computation of MPC-Based Integrated Decision Control for Autonomous Vehicles
In the paper, an integrated decision control (IDC) architecture has been introduced, seamlessly integrating autonomous decision-making and motion control into a unified processing framework. This architecture primarily comprises two key modules: a static path planner and a MPC-based dynamic optimal tracker. The former exclusively utilizes static information, such as road geometry, roadside signs, and road markings, to formulate a candidate path set. Building upon this foundation, the latter autonomously determines the most suitable driving path from the candidate paths. It integrates vehicle dynamics with dynamic information, including traffic participants and traffic lights, to design a constrained trajectory tracking controller for achieving precise motion control. Furthermore, from an engineering practice perspective, a dimension reduction control strategy for both control inputs and system constraints has been devised to enhance the real-time performance of the IDC system. Experimental results affirm that the proposed strategy effectively facilitates autonomous and secure driving of vehicles in open road traffic environments.
- 24 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 9 (11) , 7059-7069
Fast Online Computation of MPC-Based Integrated Decision Control for Autonomous Vehicles
In the paper, an integrated decision control (IDC) architecture has been introduced, seamlessly integrating autonomous decision-making and motion control into a unified processing framework. This architecture primarily comprises two key modules: a static path planner and a MPC-based dynamic optimal tracker. The former exclusively utilizes static information, such as road geometry, roadside signs, and road markings, to formulate a candidate path set. Building upon this foundation, the latter autonomously determines the most suitable driving path from the candidate paths. It integrates vehicle dynamics with dynamic information, including traffic participants and traffic lights, to design a constrained trajectory tracking controller for achieving precise motion control. Furthermore, from an engineering practice perspective, a dimension reduction control strategy for both control inputs and system constraints has been devised to enhance the real-time performance of the IDC system. Experimental results affirm that the proposed strategy effectively facilitates autonomous and secure driving of vehicles in open road traffic environments.
- 27 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 21 (3) , 4129-4142
Safe Model-Based Reinforcement Learning With an Uncertainty-Aware Reachability Certificate
Safe reinforcement learning (RL) that solves constraint-satisfactory policies provides a promising way to the broader safety-critical applications of RL in real-world problems such as robotics. Among all safe RL approaches, model-based methods reduce training time violations further due to their high sample efficiency. However, lacking safety robustness against the model uncertainties remains an issue in safe model-based RL, especially in training time safety. In this paper, we propose a distributional reachability certificate (DRC) and its Bellman equation to address model uncertainties and characterize robust persistently safe states. Furthermore, we build a safe RL framework to resolve constraints required by the DRC and its corresponding shield policy. We also devise a line search method to maintain safety and reach higher returns simultaneously while leveraging the shield policy. Comprehensive experiments on classical benchmarks such as constrained tracking and navigation indicate that the proposed algorithm achieves comparable returns with much fewer constraint violations during training. Our code is available at https://github.com/ManUtdMoon/Distributional-Reachability-Policy-Optimization. Note to Practitioners —Although it has been proven that RL can be applied in complex robotics control tasks, the training process of an RL control policy induces frequent failures because the agent needs to learn safety through constraint violations. This issue hinders the promotion of RL because a large amount of failure of robots is too expensive to afford. This paper aims to reduce the training-time violations of RL-based control methods, enabling RL to be leveraged in a broader application area. To achieve the goal, we first introduce a safety quantity describing the distribution of potential constraint violations in the long term. By imposing constraints on the quantile of the safety distribution, we can realize safety robust to the model uncertainty, which is necessary for real-world robot learning with environment uncertainty. Second, we further devise a shield policy aiming to minimize the constraint violation. The policy will intervene when the agent is about to violate state constraints, further enhancing exploration safety. Third, we implement a line search method to find an action pursuing near-optimal performance when fulfilling safety requirements strictly. Our experimental results indicate that the proposed algorithm reduces training-time violations significantly while maintaining competitive task performance. We make a step towards applying RL safely in real-world tasks. Our future work includes conducting physical verification on real robots to evaluate the algorithm and improving safety further by starting from an initially safe control policy that comes from domain knowledge.
- 27 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 21 (3) , 4129-4142
Safe Model-Based Reinforcement Learning With an Uncertainty-Aware Reachability Certificate
Safe reinforcement learning (RL) that solves constraint-satisfactory policies provides a promising way to the broader safety-critical applications of RL in real-world problems such as robotics. Among all safe RL approaches, model-based methods reduce training time violations further due to their high sample efficiency. However, lacking safety robustness against the model uncertainties remains an issue in safe model-based RL, especially in training time safety. In this paper, we propose a distributional reachability certificate (DRC) and its Bellman equation to address model uncertainties and characterize robust persistently safe states. Furthermore, we build a safe RL framework to resolve constraints required by the DRC and its corresponding shield policy. We also devise a line search method to maintain safety and reach higher returns simultaneously while leveraging the shield policy. Comprehensive experiments on classical benchmarks such as constrained tracking and navigation indicate that the proposed algorithm achieves comparable returns with much fewer constraint violations during training. Our code is available at https://github.com/ManUtdMoon/Distributional-Reachability-Policy-Optimization. Note to Practitioners —Although it has been proven that RL can be applied in complex robotics control tasks, the training process of an RL control policy induces frequent failures because the agent needs to learn safety through constraint violations. This issue hinders the promotion of RL because a large amount of failure of robots is too expensive to afford. This paper aims to reduce the training-time violations of RL-based control methods, enabling RL to be leveraged in a broader application area. To achieve the goal, we first introduce a safety quantity describing the distribution of potential constraint violations in the long term. By imposing constraints on the quantile of the safety distribution, we can realize safety robust to the model uncertainty, which is necessary for real-world robot learning with environment uncertainty. Second, we further devise a shield policy aiming to minimize the constraint violation. The policy will intervene when the agent is about to violate state constraints, further enhancing exploration safety. Third, we implement a line search method to find an action pursuing near-optimal performance when fulfilling safety requirements strictly. Our experimental results indicate that the proposed algorithm reduces training-time violations significantly while maintaining competitive task performance. We make a step towards applying RL safely in real-world tasks. Our future work includes conducting physical verification on real robots to evaluate the algorithm and improving safety further by starting from an initially safe control policy that comes from domain knowledge.
- 23 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 69 (4) , 2713-2720
Synthesizing Control Barrier Functions With Feasible Region Iteration for Safe Reinforcement Learning
Safety is a critical concern when applying reinforcement learning (RL) to real-world control problems. A widely used method for ensuring safety is to learn a control barrier function with heuristic feasibility labels that come from expert demonstrations [1] or constraint functions [2]. However, their forward invariant sets fall short of the maximum feasible region because of inaccurate labels. This paper proposes an algorithm called feasible region iteration (FRI) that learns the maximum feasible region to generate accurate feasibility labels. The core of FRI is a constraint decay function (CDF), which comes with a self-consistency condition and naturally leads to the constraint Bellman equation. The optimal CDF, which represents the maximum feasible region, is learned through the iteration of feasible region identification and feasible region expansion. Experiment results show that our algorithm achieves near-zero constraint violations and comparable or higher performance than the baselines.
- 23 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 69 (4) , 2713-2720
Synthesizing Control Barrier Functions With Feasible Region Iteration for Safe Reinforcement Learning
Safety is a critical concern when applying reinforcement learning (RL) to real-world control problems. A widely used method for ensuring safety is to learn a control barrier function with heuristic feasibility labels that come from expert demonstrations [1] or constraint functions [2]. However, their forward invariant sets fall short of the maximum feasible region because of inaccurate labels. This paper proposes an algorithm called feasible region iteration (FRI) that learns the maximum feasible region to generate accurate feasibility labels. The core of FRI is a constraint decay function (CDF), which comes with a self-consistency condition and naturally leads to the constraint Bellman equation. The optimal CDF, which represents the maximum feasible region, is learned through the iteration of feasible region identification and feasible region expansion. Experiment results show that our algorithm achieves near-zero constraint violations and comparable or higher performance than the baselines.
- 11 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 69 (1) , 426-433
Relaxed Policy Iteration Algorithm for Nonlinear Zero-Sum Games With Application to H-Infinity Control
Though policy evaluation error profoundly affects the direction of policy optimization and the convergence property, it is usually ignored in policy iteration methods. This work incorporates the practical inexact policy evaluation into a simultaneous policy update paradigm to reach the Nash equilibrium of the nonlinear zero-sum games. In the proposed algorithm, the restriction of precise policy evaluation is removed by bounded evaluation error characterized by Hamiltonian without sacrificing convergence guarantees. By exploiting Fréchet differential, the practical iterative process of value function with estimation error is converted into Newton's method with variable steps, which are inversely proportional to evaluation errors. Accordingly, we construct a monotone scalar sequence that shares the same Newton's method with the value sequence to bound the error of the value function, which enjoys an exponential convergence rate. Numerical results show its convergence in affine systems, and the potential to cope with general nonlinear plants.
- 11 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 69 (1) , 426-433
Relaxed Policy Iteration Algorithm for Nonlinear Zero-Sum Games With Application to H-Infinity Control
Though policy evaluation error profoundly affects the direction of policy optimization and the convergence property, it is usually ignored in policy iteration methods. This work incorporates the practical inexact policy evaluation into a simultaneous policy update paradigm to reach the Nash equilibrium of the nonlinear zero-sum games. In the proposed algorithm, the restriction of precise policy evaluation is removed by bounded evaluation error characterized by Hamiltonian without sacrificing convergence guarantees. By exploiting Fréchet differential, the practical iterative process of value function with estimation error is converted into Newton's method with variable steps, which are inversely proportional to evaluation errors. Accordingly, we construct a monotone scalar sequence that shares the same Newton's method with the value sequence to bound the error of the value function, which enjoys an exponential convergence rate. Numerical results show its convergence in affine systems, and the potential to cope with general nonlinear plants.