WZWenjun Zou

Wenjun Zou

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Wenjun Zou has a total of 24 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • DIK: A Diffusion Inverse Kinematics Method of Robot Manipulators

    by Jiang Wu, Wenjun Zou, Yinsong Ma, Haoqi Yan, Junjie Zhao, Yinuo Wang, Shengbo Eben Li, Jingliang Duan

    Inverse kinematics (IK) determines the joint configurations needed to achieve a specific end pose of a kinematic chain. IK solutions for complex robot manipulators are typically nonunique, allowing multiple joint configurations to correspond to the same end pose. However, traditional IK methods generally yield only a single solution. To overcome this limitation, this study introduces the diffusion inverse kinematics (DIK) method, which can provide multiple IK solutions for the same end pose. This capability allows for the selection of the optimal configuration based on criteria, such as accuracy, proximity to the current position, or adherence to specific constraints. DIK utilizes a diffusion generation process akin to the sampling process of the diffusion model to generate these multimodal solutions, and it employs a self-supervised learning framework to align the training data with real-world applications. This framework updates the parameters of the diffusion generation process by minimizing the difference between the actual poses and the target poses. For robot manipulators that operate under specific constraints, DIK integrates these constraints into the IK solution process, and addresses them using exterior point methods. Experimental results affirm that DIK not only generates multimodal IK solutions but also achieves high accuracy and zero-collision outcomes.

  • Multimodal Reinforcement Learning With Score-Based Policy

    by Wenjun Zou, Yinuo Wang, Tong Liu, Bin Shuai, Liming Xiao, Yinsong Ma, Jingliang Duan, Shengbo Eben Li

    Learning multimodal policies is crucial for enhancing exploration in online reinforcement learning (RL), especially in tasks with continuous action spaces and non-convex reward landscapes. While recent diffusion policies show promise, they often suffer from low computational efficiency in online settings. A more training-efficient paradigm involves modeling the policy as a Boltzmann distribution and guiding the action sampling directly with the gradient of the Q-value with respect to the action (proportional to the score function of the policy), such as via Langevin dynamics. However, analysis in this paper reveals that this gradient-guided approach suffers from two critical challenges: sampling instability caused by the widely varying magnitude of action gradients; and mode imbalance, where the sampling process inaccurately represents the weights of different high-value action modes. To address these challenges, this paper introduces three targeted techniques: score normalization and reshaping to stabilize the sampling process, and value-based resampling to correct mode imbalance. These techniques are then integrated into an actor-critic framework, resulting in the Score-Enhanced Actor-Critic (SEAC) algorithm. Simulation and real-world experiments demonstrate that SEAC not only effectively learns multimodal behaviors but also achieves state-of-the-art performance and high computational efficiency compared to prior multimodal RL methods. The code of this paper is available at https://github.com/THUzouwenjun/SEAC.


  • Fast Online Computation of MPC-Based Integrated Decision Control for Autonomous Vehicles

    by Shaosong Li, Wenjun Zou, Jiaxin Gao, Yuming Yin, Dongyoon Kim, Sen Yang, Shengbo Eben Li

    In the paper, an integrated decision control (IDC) architecture has been introduced, seamlessly integrating autonomous decision-making and motion control into a unified processing framework. This architecture primarily comprises two key modules: a static path planner and a MPC-based dynamic optimal tracker. The former exclusively utilizes static information, such as road geometry, roadside signs, and road markings, to formulate a candidate path set. Building upon this foundation, the latter autonomously determines the most suitable driving path from the candidate paths. It integrates vehicle dynamics with dynamic information, including traffic participants and traffic lights, to design a constrained trajectory tracking controller for achieving precise motion control. Furthermore, from an engineering practice perspective, a dimension reduction control strategy for both control inputs and system constraints has been devised to enhance the real-time performance of the IDC system. Experimental results affirm that the proposed strategy effectively facilitates autonomous and secure driving of vehicles in open road traffic environments.


  • Safe Model-Based Reinforcement Learning With an Uncertainty-Aware Reachability Certificate

    by Dongjie Yu, Wenjun Zou, Yujie Yang, Haitong Ma, Shengbo Eben Li, Yuming Yin, Jianyu Chen, Jingliang Duan

    Safe reinforcement learning (RL) that solves constraint-satisfactory policies provides a promising way to the broader safety-critical applications of RL in real-world problems such as robotics. Among all safe RL approaches, model-based methods reduce training time violations further due to their high sample efficiency. However, lacking safety robustness against the model uncertainties remains an issue in safe model-based RL, especially in training time safety. In this paper, we propose a distributional reachability certificate (DRC) and its Bellman equation to address model uncertainties and characterize robust persistently safe states. Furthermore, we build a safe RL framework to resolve constraints required by the DRC and its corresponding shield policy. We also devise a line search method to maintain safety and reach higher returns simultaneously while leveraging the shield policy. Comprehensive experiments on classical benchmarks such as constrained tracking and navigation indicate that the proposed algorithm achieves comparable returns with much fewer constraint violations during training. Our code is available at https://github.com/ManUtdMoon/Distributional-Reachability-Policy-Optimization. Note to Practitioners —Although it has been proven that RL can be applied in complex robotics control tasks, the training process of an RL control policy induces frequent failures because the agent needs to learn safety through constraint violations. This issue hinders the promotion of RL because a large amount of failure of robots is too expensive to afford. This paper aims to reduce the training-time violations of RL-based control methods, enabling RL to be leveraged in a broader application area. To achieve the goal, we first introduce a safety quantity describing the distribution of potential constraint violations in the long term. By imposing constraints on the quantile of the safety distribution, we can realize safety robust to the model uncertainty, which is necessary for real-world robot learning with environment uncertainty. Second, we further devise a shield policy aiming to minimize the constraint violation. The policy will intervene when the agent is about to violate state constraints, further enhancing exploration safety. Third, we implement a line search method to find an action pursuing near-optimal performance when fulfilling safety requirements strictly. Our experimental results indicate that the proposed algorithm reduces training-time violations significantly while maintaining competitive task performance. We make a step towards applying RL safely in real-world tasks. Our future work includes conducting physical verification on real robots to evaluate the algorithm and improving safety further by starting from an initially safe control policy that comes from domain knowledge.


  • Synthesizing Control Barrier Functions With Feasible Region Iteration for Safe Reinforcement Learning

    by Yujie Yang, Yuhang Zhang, Wenjun Zou, Jianyu Chen, Yuming Yin, Shengbo Eben Li

    Safety is a critical concern when applying reinforcement learning (RL) to real-world control problems. A widely used method for ensuring safety is to learn a control barrier function with heuristic feasibility labels that come from expert demonstrations [1] or constraint functions [2]. However, their forward invariant sets fall short of the maximum feasible region because of inaccurate labels. This paper proposes an algorithm called feasible region iteration (FRI) that learns the maximum feasible region to generate accurate feasibility labels. The core of FRI is a constraint decay function (CDF), which comes with a self-consistency condition and naturally leads to the constraint Bellman equation. The optimal CDF, which represents the maximum feasible region, is learned through the iteration of feasible region identification and feasible region expansion. Experiment results show that our algorithm achieves near-zero constraint violations and comparable or higher performance than the baselines.


  • Relaxed Policy Iteration Algorithm for Nonlinear Zero-Sum Games With Application to H-Infinity Control

    by Jie Li, Shengbo Eben Li, Jingliang Duan, Yao Lyu, Wenjun Zou, Yang Guan, Yuming Yin

    Though policy evaluation error profoundly affects the direction of policy optimization and the convergence property, it is usually ignored in policy iteration methods. This work incorporates the practical inexact policy evaluation into a simultaneous policy update paradigm to reach the Nash equilibrium of the nonlinear zero-sum games. In the proposed algorithm, the restriction of precise policy evaluation is removed by bounded evaluation error characterized by Hamiltonian without sacrificing convergence guarantees. By exploiting Fréchet differential, the practical iterative process of value function with estimation error is converted into Newton's method with variable steps, which are inversely proportional to evaluation errors. Accordingly, we construct a monotone scalar sequence that shares the same Newton's method with the value sequence to bound the error of the value function, which enjoys an exponential convergence rate. Numerical results show its convergence in affine systems, and the potential to cope with general nonlinear plants.