Yangang Ren
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yangang Ren has a total of 45 co-authors. Here are the top co-authors by number of shared publications:
- Shengbo Eben Li
Tsinghua University
14 shared publications - Jingliang Duan
School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China
9 shared publications - Yang Guan
School of Vehicle and Mobility, Tsinghua University, Beijing, China
7 shared publications - Keqiang Li
Tsinghua University
5 shared publications - Bo Cheng
School of Vehicle and Mobility, Tsinghua University, Beijing, China
4 shared publications - Jie Li
College of Mechanical and Vehicle Engineering, Chongqing University, Chongqing, China
4 shared publications
Latest Publications
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 27 (7) , 8031-8042
Learning Optimal Robust Control for Nonlinear Mixed Traffic Under External Disturbance
The integration of connected and automated vehicles (CAVs) into traffic systems holds potential to mitigate undesired disturbances. Nevertheless, coexisting human-driven vehicles (HDVs) introduce complex behavioral disturbances, which has imposed critical challenges for control robustness. This study develops a computational framework based on policy iteration to derive robust control policies with optimized attenuation performance for nonlinear mixed traffic flow. Specifically, robust $H_{\infty }$ control problem is solved by applying the framework of zero-sum game, whose solution at the Nash equilibrium is transformed into a Hamilton–Jacobi (HJ) inequality with a Hamiltonian constraint. For achieving desired attenuation performance, the value function is updated by gradient descent based on counterexamples violating Hamiltonian and monotonicity constraints, where the positive definiteness of the value function is ensured by convex neural networks, facilitating the analysis of control stability via Lyapunov methods. By utilizing constraint gaps, the attenuation level is optimized through the analytical formulae derived from the HJ inequality. The stability and algorithm convergence are proved. Experimental results demonstrate the capability of the learned controller to effectively attenuate disturbance propagation and stabilize mixed traffic flow.
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 27 (7) , 8031-8042
Learning Optimal Robust Control for Nonlinear Mixed Traffic Under External Disturbance
The integration of connected and automated vehicles (CAVs) into traffic systems holds potential to mitigate undesired disturbances. Nevertheless, coexisting human-driven vehicles (HDVs) introduce complex behavioral disturbances, which has imposed critical challenges for control robustness. This study develops a computational framework based on policy iteration to derive robust control policies with optimized attenuation performance for nonlinear mixed traffic flow. Specifically, robust $H_{\infty }$ control problem is solved by applying the framework of zero-sum game, whose solution at the Nash equilibrium is transformed into a Hamilton–Jacobi (HJ) inequality with a Hamiltonian constraint. For achieving desired attenuation performance, the value function is updated by gradient descent based on counterexamples violating Hamiltonian and monotonicity constraints, where the positive definiteness of the value function is ensured by convex neural networks, facilitating the analysis of control stability via Lyapunov methods. By utilizing constraint gaps, the attenuation level is optimized through the analytical formulae derived from the HJ inequality. The stability and algorithm convergence are proved. Experimental results demonstrate the capability of the learned controller to effectively attenuate disturbance propagation and stabilize mixed traffic flow.
- 24 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 11 (4) , 545-553
Learn a Robust Policy for Real-World Driving With Adversarial Reinforcement Learning
Reinforcement learning (RL) has emerged as a promising approach for achieving high-level autonomous driving as its self-evolving ability without reliance on human rules. Although RL-driven methods have yielded fruitful demonstrations in driving domain, most of them are trained and validated only on simulation platforms. Due to the inherent difference between simulation and reality, the driving policy usually performs poorly when applied to a realistic environment. In this paper, we propose an adversarial training framework for driving policy to enhance practical performance, which introduces the adversarial policy to simulate the discrepancy of environments during training and incorporates the collected data from the real world to provide the discrepancy bound. Besides, the adversarial policy is iteratively updated with gradient-based optimization, which enables the automatic generation of diversified discrepancies for different traffic participants. The action projection is developed to ensure that the output of adversary satisfies the discrepancy bound given by data, so as to prevent an aggressive adversary making the overly conservative policy. We evaluate the trained policy on a fully sized vehicle at an urban intersection with mixed traffic flows. Results indicate that our driving policy can handle unseen behaviors of traffic actors meanwhile realizing the safe and smooth control for the automated vehicle. Our work provides a feasible solution for RL implementation in the field of real-world autonomous driving.
- 24 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 11 (4) , 545-553
Learn a Robust Policy for Real-World Driving With Adversarial Reinforcement Learning
Reinforcement learning (RL) has emerged as a promising approach for achieving high-level autonomous driving as its self-evolving ability without reliance on human rules. Although RL-driven methods have yielded fruitful demonstrations in driving domain, most of them are trained and validated only on simulation platforms. Due to the inherent difference between simulation and reality, the driving policy usually performs poorly when applied to a realistic environment. In this paper, we propose an adversarial training framework for driving policy to enhance practical performance, which introduces the adversarial policy to simulate the discrepancy of environments during training and incorporates the collected data from the real world to provide the discrepancy bound. Besides, the adversarial policy is iteratively updated with gradient-based optimization, which enables the automatic generation of diversified discrepancies for different traffic participants. The action projection is developed to ensure that the output of adversary satisfies the discrepancy bound given by data, so as to prevent an aggressive adversary making the overly conservative policy. We evaluate the trained policy on a fully sized vehicle at an urban intersection with mixed traffic flows. Results indicate that our driving policy can handle unseen behaviors of traffic actors meanwhile realizing the safe and smooth control for the automated vehicle. Our work provides a feasible solution for RL implementation in the field of real-world autonomous driving.
- 19 February 2025
- Published by Elsevier in Neurocomputing
- Vol. 630
Smooth policy iteration for zero-sum Markov Games
- 19 February 2025
- Published by Elsevier in Neurocomputing
- Vol. 630
Smooth policy iteration for zero-sum Markov Games
- 5 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computational Intelligence Magazine
- Vol. 19 (2) , 96-112
Encoding Distributional Soft Actor-Critic for Autonomous Driving in Multi-Lane Scenarios [Research Frontier]
This paper proposes a new reinforcement learning (RL) algorithm, called encoding distributional soft actor-critic (E-DSAC), for decision-making in autonomous driving. Unlike existing RL-based decision-making methods, E-DSAC is suitable for situations where the number of surrounding vehicles is variable and eliminates the requirement for manually pre-designed sorting rules, resulting in higher policy performance and generality. Firstly, an encoding distributional policy iteration (DPI) framework is developed by embedding a permutation invariant module, which employs a feature neural network (NN) to encode the indicators of each vehicle, in the distributional RL framework. The proposed DPI framework is proven to exhibit important properties in terms of convergence and global optimality. Next, based on the developed encoding DPI framework, the E-DSAC algorithm is proposed by adding the gradient-based update rule of the feature NN to the policy evaluation process of the DSAC algorithm. Then, the multi-lane driving task and the corresponding reward function are designed to verify the effectiveness of the proposed algorithm. Results show that the policy learned by E-DSAC can realize efficient, smooth, and relatively safe autonomous driving in the designed scenario. And the final policy performance attained by E-DSAC surpasses that of DSAC by approximately threefold. Furthermore, its effectiveness has also been verified in real vehicle experiments.
- 5 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computational Intelligence Magazine
- Vol. 19 (2) , 96-112
Encoding Distributional Soft Actor-Critic for Autonomous Driving in Multi-Lane Scenarios [Research Frontier]
This paper proposes a new reinforcement learning (RL) algorithm, called encoding distributional soft actor-critic (E-DSAC), for decision-making in autonomous driving. Unlike existing RL-based decision-making methods, E-DSAC is suitable for situations where the number of surrounding vehicles is variable and eliminates the requirement for manually pre-designed sorting rules, resulting in higher policy performance and generality. Firstly, an encoding distributional policy iteration (DPI) framework is developed by embedding a permutation invariant module, which employs a feature neural network (NN) to encode the indicators of each vehicle, in the distributional RL framework. The proposed DPI framework is proven to exhibit important properties in terms of convergence and global optimality. Next, based on the developed encoding DPI framework, the E-DSAC algorithm is proposed by adding the gradient-based update rule of the feature NN to the policy evaluation process of the DSAC algorithm. Then, the multi-lane driving task and the corresponding reward function are designed to verify the effectiveness of the proposed algorithm. Results show that the policy learned by E-DSAC can realize efficient, smooth, and relatively safe autonomous driving in the designed scenario. And the final policy performance attained by E-DSAC surpasses that of DSAC by approximately threefold. Furthermore, its effectiveness has also been verified in real vehicle experiments.
- Open Access
- 10 October 2022
- Published by International Society for Terrain-Vehicle Systems
Steadily Learn to Drive with Virtual Memory
Paper presented at the 11th Asia-Pacific Regional Conference of the ISTVS
- 10 October 2022
- Published by International Society for Terrain-Vehicle Systems
Open AccessSteadily Learn to Drive with Virtual Memory
Paper presented at the 11th Asia-Pacific Regional Conference of the ISTVS
- 16 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (12) , 24145-24156
Self-Learned Intelligence for Integrated Decision and Control of Automated Vehicles at Signalized Intersections
Intersection is one of the most accident-prone urban scenarios for autonomous driving wherein making safe and computationally efficient decisions is non-trivial. Current research mainly focuses on the simplified traffic conditions while ignoring the existence of mixed traffic flows, i.e., vehicles, cyclists and pedestrians. For urban roads, different participants lead to a quite dynamic and complex interaction, posing great difficulty to learn an intelligent policy. This paper develops the dynamic permutation state representation in the framework of integrated decision and control (IDC) to handle signalized intersections with mixed traffic flows. Specially, this representation introduces an encoding function and summation operator to construct driving states from environmental observation, capable of dealing with different types and variant number of traffic participants. A constrained optimal control problem is built wherein the objective involves tracking performance and the constraints for different participants, roads and signal lights are designed respectively to assure safety. We solve this problem by gradient-based optimization, wherein the reasonable state will be given by the encoding function and then served as the input of policy and value function. An off-policy training is designed to reuse observations from driving environment and backpropagation through time is utilized to update the policy function and encoding function jointly. Verification result shows that the dynamic permutation state representation can enhance the driving performance of IDC, including comfort, decision compliance and safety with a large margin. The trained driving policy can realize efficient and smooth passing in the complex intersection, guaranteeing driving intelligence and safety simultaneously.
- 16 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (12) , 24145-24156
Self-Learned Intelligence for Integrated Decision and Control of Automated Vehicles at Signalized Intersections
Intersection is one of the most accident-prone urban scenarios for autonomous driving wherein making safe and computationally efficient decisions is non-trivial. Current research mainly focuses on the simplified traffic conditions while ignoring the existence of mixed traffic flows, i.e., vehicles, cyclists and pedestrians. For urban roads, different participants lead to a quite dynamic and complex interaction, posing great difficulty to learn an intelligent policy. This paper develops the dynamic permutation state representation in the framework of integrated decision and control (IDC) to handle signalized intersections with mixed traffic flows. Specially, this representation introduces an encoding function and summation operator to construct driving states from environmental observation, capable of dealing with different types and variant number of traffic participants. A constrained optimal control problem is built wherein the objective involves tracking performance and the constraints for different participants, roads and signal lights are designed respectively to assure safety. We solve this problem by gradient-based optimization, wherein the reasonable state will be given by the encoding function and then served as the input of policy and value function. An off-policy training is designed to reuse observations from driving environment and backpropagation through time is utilized to update the policy function and encoding function jointly. Verification result shows that the dynamic permutation state representation can enhance the driving performance of IDC, including comfort, decision compliance and safety with a large margin. The trained driving policy can realize efficient and smooth passing in the complex intersection, guaranteeing driving intelligence and safety simultaneously.
- Top Cited Papers
- 19 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 53 (2) , 859-873
Integrated Decision and Control: Toward Interpretable and Computationally Efficient Driving Intelligence
Decision and control are core functionalities of high-level automated vehicles. Current mainstream methods, such as functional decomposition and end-to-end reinforcement learning (RL), suffer high time complexity or poor interpretability and adaptability on real-world autonomous driving tasks. In this article, we present an interpretable and computationally efficient framework called integrated decision and control (IDC) for automated vehicles, which decomposes the driving task into static path planning and dynamic optimal tracking that are structured hierarchically. First, the static path planning generates several candidate paths only considering static traffic elements. Then, the dynamic optimal tracking is designed to track the optimal path while considering the dynamic obstacles. To that end, we formulate a constrained optimal control problem (OCP) for each candidate path, optimize them separately, and follow the one with the best tracking performance. To unload the heavy online computation, we propose a model-based RL algorithm that can be served as an approximate-constrained OCP solver. Specifically, the OCPs for all paths are considered together to construct a single complete RL problem and then solved offline in the form of value and policy networks for real-time online path selecting and tracking, respectively. We verify our framework in both simulations and the real world. Results show that compared with baseline methods, IDC has an order of magnitude higher online computing efficiency, as well as better driving performance, including traffic efficiency and safety. In addition, it yields great interpretability and adaptability among different driving scenarios and tasks.
- 19 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 53 (2) , 859-873
Top Cited PapersIntegrated Decision and Control: Toward Interpretable and Computationally Efficient Driving Intelligence
Decision and control are core functionalities of high-level automated vehicles. Current mainstream methods, such as functional decomposition and end-to-end reinforcement learning (RL), suffer high time complexity or poor interpretability and adaptability on real-world autonomous driving tasks. In this article, we present an interpretable and computationally efficient framework called integrated decision and control (IDC) for automated vehicles, which decomposes the driving task into static path planning and dynamic optimal tracking that are structured hierarchically. First, the static path planning generates several candidate paths only considering static traffic elements. Then, the dynamic optimal tracking is designed to track the optimal path while considering the dynamic obstacles. To that end, we formulate a constrained optimal control problem (OCP) for each candidate path, optimize them separately, and follow the one with the best tracking performance. To unload the heavy online computation, we propose a model-based RL algorithm that can be served as an approximate-constrained OCP solver. Specifically, the OCPs for all paths are considered together to construct a single complete RL problem and then solved offline in the form of value and policy networks for real-time online path selecting and tracking, respectively. We verify our framework in both simulations and the real world. Results show that compared with baseline methods, IDC has an order of magnitude higher online computing efficiency, as well as better driving performance, including traffic efficiency and safety. In addition, it yields great interpretability and adaptability among different driving scenarios and tasks.
- 9 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (9) , 15980-15992
Indirect Shared Control Through Non-Zero Sum Differential Game for Cooperative Automated Driving
Cooperative driving of human driver and automated system can effectively reduce the necessity of extremely accurate environment perception of highly automated vehicles, and enhance the robustness of decision-making and motion control. However, due to the two players’ different intentions, severe conflicts may exist during the cooperation, which often result in negative consequences on driving safety and maneuverability. This paper presents an indirect shared control method to model the situation and improve the driving performance, which focus on the affine input nonlinear vehicle dynamic system for shared controller design under the framework of non-zero sum differential game. The Nash equilibria strategy indicates the best response for the automated system, which can guide the automated controller to act more safely and comfortably. Aimed to obtain fast solution for practical application, approximate dynamic programming is utilized to find the Nash equilibria, which is represented by deep neural networks and solved iteratively. Driver-in-the-loop tests on a driving simulator were conducted to verify the performance of the proposed method under highway driving scenarios. The results show that the designed controller is able to reduce the driving workload and ensure the driving safety.
- 9 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (9) , 15980-15992
Indirect Shared Control Through Non-Zero Sum Differential Game for Cooperative Automated Driving
Cooperative driving of human driver and automated system can effectively reduce the necessity of extremely accurate environment perception of highly automated vehicles, and enhance the robustness of decision-making and motion control. However, due to the two players’ different intentions, severe conflicts may exist during the cooperation, which often result in negative consequences on driving safety and maneuverability. This paper presents an indirect shared control method to model the situation and improve the driving performance, which focus on the affine input nonlinear vehicle dynamic system for shared controller design under the framework of non-zero sum differential game. The Nash equilibria strategy indicates the best response for the automated system, which can guide the automated controller to act more safely and comfortably. Aimed to obtain fast solution for practical application, approximate dynamic programming is utilized to find the Nash equilibria, which is represented by deep neural networks and solved iteratively. Driver-in-the-loop tests on a driving simulator were conducted to verify the performance of the proposed method under highway driving scenarios. The results show that the designed controller is able to reduce the driving workload and ensure the driving safety.