YRYangang Ren

Yangang Ren

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Yangang Ren has a total of 45 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Learning Optimal Robust Control for Nonlinear Mixed Traffic Under External Disturbance

    by Jie Li, Jiawei Wang, Yangang Ren, Yujie Yang, Shen Li, Guofa Li, Shengbo Eben Li

    The integration of connected and automated vehicles (CAVs) into traffic systems holds potential to mitigate undesired disturbances. Nevertheless, coexisting human-driven vehicles (HDVs) introduce complex behavioral disturbances, which has imposed critical challenges for control robustness. This study develops a computational framework based on policy iteration to derive robust control policies with optimized attenuation performance for nonlinear mixed traffic flow. Specifically, robust $H_{\infty }$ control problem is solved by applying the framework of zero-sum game, whose solution at the Nash equilibrium is transformed into a Hamilton–Jacobi (HJ) inequality with a Hamiltonian constraint. For achieving desired attenuation performance, the value function is updated by gradient descent based on counterexamples violating Hamiltonian and monotonicity constraints, where the positive definiteness of the value function is ensured by convex neural networks, facilitating the analysis of control stability via Lyapunov methods. By utilizing constraint gaps, the attenuation level is optimized through the analytical formulae derived from the HJ inequality. The stability and algorithm convergence are proved. Experimental results demonstrate the capability of the learned controller to effectively attenuate disturbance propagation and stabilize mixed traffic flow.

  • Learn a Robust Policy for Real-World Driving With Adversarial Reinforcement Learning

    by Yangang Ren, Guojian Zhan, Shengbo Eben Li, Jingliang Duan, Yao Lyu, Yang Guan, Keqiang Li

    Reinforcement learning (RL) has emerged as a promising approach for achieving high-level autonomous driving as its self-evolving ability without reliance on human rules. Although RL-driven methods have yielded fruitful demonstrations in driving domain, most of them are trained and validated only on simulation platforms. Due to the inherent difference between simulation and reality, the driving policy usually performs poorly when applied to a realistic environment. In this paper, we propose an adversarial training framework for driving policy to enhance practical performance, which introduces the adversarial policy to simulate the discrepancy of environments during training and incorporates the collected data from the real world to provide the discrepancy bound. Besides, the adversarial policy is iteratively updated with gradient-based optimization, which enables the automatic generation of diversified discrepancies for different traffic participants. The action projection is developed to ensure that the output of adversary satisfies the discrepancy bound given by data, so as to prevent an aggressive adversary making the overly conservative policy. We evaluate the trained policy on a fully sized vehicle at an urban intersection with mixed traffic flows. Results indicate that our driving policy can handle unseen behaviors of traffic actors meanwhile realizing the safe and smooth control for the automated vehicle. Our work provides a feasible solution for RL implementation in the field of real-world autonomous driving.


  • Smooth policy iteration for zero-sum Markov Games

    by Yangang Ren, Yao Lyu, Wenxuan Wang, Shengbo Eben Li, Zeyang Li, Jingliang Duan


  • Encoding Distributional Soft Actor-Critic for Autonomous Driving in Multi-Lane Scenarios [Research Frontier]

    by Jingliang Duan, Yangang Ren, Fawang Zhang, Jie Li, Shengbo Eben Li, Yang Guan, Keqiang Li

    This paper proposes a new reinforcement learning (RL) algorithm, called encoding distributional soft actor-critic (E-DSAC), for decision-making in autonomous driving. Unlike existing RL-based decision-making methods, E-DSAC is suitable for situations where the number of surrounding vehicles is variable and eliminates the requirement for manually pre-designed sorting rules, resulting in higher policy performance and generality. Firstly, an encoding distributional policy iteration (DPI) framework is developed by embedding a permutation invariant module, which employs a feature neural network (NN) to encode the indicators of each vehicle, in the distributional RL framework. The proposed DPI framework is proven to exhibit important properties in terms of convergence and global optimality. Next, based on the developed encoding DPI framework, the E-DSAC algorithm is proposed by adding the gradient-based update rule of the feature NN to the policy evaluation process of the DSAC algorithm. Then, the multi-lane driving task and the corresponding reward function are designed to verify the effectiveness of the proposed algorithm. Results show that the policy learned by E-DSAC can realize efficient, smooth, and relatively safe autonomous driving in the designed scenario. And the final policy performance attained by E-DSAC surpasses that of DSAC by approximately threefold. Furthermore, its effectiveness has also been verified in real vehicle experiments.


  • Improve generalization of driving policy at signalized intersections with adversarial learning

    by Yangang Ren, Guojian Zhan, Liye Tang, Shengbo Eben Li, Jianhua Jiang, Keqiang Li, Jingliang Duan


  • Open Access

    Steadily Learn to Drive with Virtual Memory

    by Yuhang Zhang, Yao Mu, Shengbo Li, Yangang Ren, Liye Tang, Yujie Yang, Chen Chen

    Paper presented at the 11th Asia-Pacific Regional Conference of the ISTVS


  • Self-Learned Intelligence for Integrated Decision and Control of Automated Vehicles at Signalized Intersections

    by Yangang Ren, Jianhua Jiang, Guojian Zhan, Shengbo Eben Li, Chen Chen, Keqiang Li, Jingliang Duan

    Intersection is one of the most accident-prone urban scenarios for autonomous driving wherein making safe and computationally efficient decisions is non-trivial. Current research mainly focuses on the simplified traffic conditions while ignoring the existence of mixed traffic flows, i.e., vehicles, cyclists and pedestrians. For urban roads, different participants lead to a quite dynamic and complex interaction, posing great difficulty to learn an intelligent policy. This paper develops the dynamic permutation state representation in the framework of integrated decision and control (IDC) to handle signalized intersections with mixed traffic flows. Specially, this representation introduces an encoding function and summation operator to construct driving states from environmental observation, capable of dealing with different types and variant number of traffic participants. A constrained optimal control problem is built wherein the objective involves tracking performance and the constraints for different participants, roads and signal lights are designed respectively to assure safety. We solve this problem by gradient-based optimization, wherein the reasonable state will be given by the encoding function and then served as the input of policy and value function. An off-policy training is designed to reuse observations from driving environment and backpropagation through time is utilized to update the policy function and encoding function jointly. Verification result shows that the dynamic permutation state representation can enhance the driving performance of IDC, including comfort, decision compliance and safety with a large margin. The trained driving policy can realize efficient and smooth passing in the complex intersection, guaranteeing driving intelligence and safety simultaneously.


  • Top Cited Papers

    Integrated Decision and Control: Toward Interpretable and Computationally Efficient Driving Intelligence

    by Yang Guan, Yangang Ren, Qi Sun, Shengbo Eben Li, Haitong Ma, Jingliang Duan, Yifan Dai, Bo Cheng

    Decision and control are core functionalities of high-level automated vehicles. Current mainstream methods, such as functional decomposition and end-to-end reinforcement learning (RL), suffer high time complexity or poor interpretability and adaptability on real-world autonomous driving tasks. In this article, we present an interpretable and computationally efficient framework called integrated decision and control (IDC) for automated vehicles, which decomposes the driving task into static path planning and dynamic optimal tracking that are structured hierarchically. First, the static path planning generates several candidate paths only considering static traffic elements. Then, the dynamic optimal tracking is designed to track the optimal path while considering the dynamic obstacles. To that end, we formulate a constrained optimal control problem (OCP) for each candidate path, optimize them separately, and follow the one with the best tracking performance. To unload the heavy online computation, we propose a model-based RL algorithm that can be served as an approximate-constrained OCP solver. Specifically, the OCPs for all paths are considered together to construct a single complete RL problem and then solved offline in the form of value and policy networks for real-time online path selecting and tracking, respectively. We verify our framework in both simulations and the real world. Results show that compared with baseline methods, IDC has an order of magnitude higher online computing efficiency, as well as better driving performance, including traffic efficiency and safety. In addition, it yields great interpretability and adaptability among different driving scenarios and tasks.


  • Indirect Shared Control Through Non-Zero Sum Differential Game for Cooperative Automated Driving

    by Wenyu Li, Qingkun Li, Shengbo Eben Li, Renjie Li, Yangang Ren, Wenjun Wang

    Cooperative driving of human driver and automated system can effectively reduce the necessity of extremely accurate environment perception of highly automated vehicles, and enhance the robustness of decision-making and motion control. However, due to the two players’ different intentions, severe conflicts may exist during the cooperation, which often result in negative consequences on driving safety and maneuverability. This paper presents an indirect shared control method to model the situation and improve the driving performance, which focus on the affine input nonlinear vehicle dynamic system for shared controller design under the framework of non-zero sum differential game. The Nash equilibria strategy indicates the best response for the automated system, which can guide the automated controller to act more safely and comfortably. Aimed to obtain fast solution for practical application, approximate dynamic programming is utilized to find the Nash equilibria, which is represented by deep neural networks and solved iteratively. Driver-in-the-loop tests on a driving simulator were conducted to verify the performance of the proposed method under highway driving scenarios. The results show that the designed controller is able to reduce the driving workload and ensure the driving safety.