Yupeng Zheng
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Yupeng Zheng has a total of 99 co-authors. Here are the top co-authors by number of shared publications:
- Dongbin Zhao
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
8 shared publications - Qichao Zhang
MAIS-SKL, CASIA, Beijing, China
8 shared publications - Songen Gu
School of Data Science, Fudan University, Shanghai, China
4 shared publications - Yuhang Zheng
SMEA, Beihang University and EncoSmart, Haidian, China
4 shared publications - Bu Jin
Institute of Automation, Chinese Academy of Sciences (CASIA), Haidian, China
3 shared publications - Deheng Qian
Chongqing Chang'an Technology Co., Ltd., Beijing, China
3 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 12271-12280
DriveVLA-M0: Failure-Aware Memory Augmentation for Autonomous Driving
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 12271-12280
Open AccessDriveVLA-M0: Failure-Aware Memory Augmentation for Autonomous Driving
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 10306-10315
Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 10306-10315
Open AccessLatent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
- 1 October 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (10) , 12240-12247
World4RL: Diffusion World Models for Policy Refinement With Reinforcement Learning for Robotic Manipulation
- 1 October 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (10) , 12240-12247
World4RL: Diffusion World Models for Policy Refinement With Reinforcement Learning for Robotic Manipulation
- 12 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. PP (21682216) , 1-14
SoAD: Safety-Oriented Value Estimation for Enhanced Closed-Loop End-to-End Autonomous Driving
End-to-end (E2E) autonomous driving systems, which map sensory inputs directly to vehicle planning, have garnered attention for harnessing the potential of data-driven methodologies in motion planning. However, current methods face two limitations that undermine their safety performance in closed-loop driving tasks. First, the predominant imitation learning (IL) paradigm overlooks long-term safety beyond predefined planning horizons, potentially guiding the ego vehicle into hazardous states. Second, the lack of reliable online evaluation mechanisms limits real-time responses to safety risks. To overcome these challenges, we propose SoAD, a safety-oriented E2E framework that integrates long-term safety awareness into planning. SoAD is distinguished by a reinforcement learning (RL)-based value estimation module to quantify the safety of planned trajectories, and a vector world model (VWM) to generate interaction-aware future rollouts. During training, the system benefits from value-guided fine-tuning (VFT) that optimizes the planning distribution to favor safer trajectories. In closed-loop deployment, a planning rescoring (PRS) mechanism is designed to perform reliable online evaluation by combining ego-conditional predictions from the VWM with corresponding safety value estimates. Experimental results on the Bench2Drive closed-loop benchmark demonstrate the state-of-the-art (SoTA) performance of SoAD, achieving a 66.40% improvement in driving score (DS) compared to the vectorized scene representation for efficient autonomous driving (VAD) baseline, while zero-shot evaluation on the driving in occlusion simulation (DOS) benchmark further highlights its strong generalization ability.
- 12 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. PP (21682216) , 1-14
SoAD: Safety-Oriented Value Estimation for Enhanced Closed-Loop End-to-End Autonomous Driving
End-to-end (E2E) autonomous driving systems, which map sensory inputs directly to vehicle planning, have garnered attention for harnessing the potential of data-driven methodologies in motion planning. However, current methods face two limitations that undermine their safety performance in closed-loop driving tasks. First, the predominant imitation learning (IL) paradigm overlooks long-term safety beyond predefined planning horizons, potentially guiding the ego vehicle into hazardous states. Second, the lack of reliable online evaluation mechanisms limits real-time responses to safety risks. To overcome these challenges, we propose SoAD, a safety-oriented E2E framework that integrates long-term safety awareness into planning. SoAD is distinguished by a reinforcement learning (RL)-based value estimation module to quantify the safety of planned trajectories, and a vector world model (VWM) to generate interaction-aware future rollouts. During training, the system benefits from value-guided fine-tuning (VFT) that optimizes the planning distribution to favor safer trajectories. In closed-loop deployment, a planning rescoring (PRS) mechanism is designed to perform reliable online evaluation by combining ego-conditional predictions from the VWM with corresponding safety value estimates. Experimental results on the Bench2Drive closed-loop benchmark demonstrate the state-of-the-art (SoTA) performance of SoAD, achieving a 66.40% improvement in driving score (DS) compared to the vectorized scene representation for efficient autonomous driving (VAD) baseline, while zero-shot evaluation on the driving in occlusion simulation (DOS) benchmark further highlights its strong generalization ability.
- Top Cited Papers
- 12 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. PP (23798920) , 1-14
PlanAgent: A Multi-modal Large Language Agent for Closed-loop Vehicle Motion Planning
Vehicle motion planning is an essential component of autonomous driving technology. Current rule-based vehicle motion planning methods perform satisfactorily in common scenarios but struggle to generalize to long-tailed situations. Meanwhile, learning-based methods have yet to achieve superior performance over rule-based approaches in large-scale closed-loop scenarios. To address these issues, we propose PlanAgent, the first mid-to-mid planning system based on a multi-modal large language model (MLLM). MLLM is used as a cognitive agent to introduce human-like knowledge, interpretability, and commonsense reasoning into closed-loop planning. Specifically, PlanAgent leverages the power of MLLM through three core modules. First, an Environment Transformation module constructs a Bird’s Eye View (BEV) map and a lane-graph-based textual description from the environment as inputs. Second, a Reasoning Engine module introduces a hierarchical chain-of-thought from scene understanding to lateral and longitudinal motion instructions, culminating in planner code generation. Last, a Reflection module is integrated to simulate and evaluate the generated planner for reducing MLLM’s uncertainty. PlanAgent is endowed with the common-sense reasoning and generalization capability of MLLM, which empowers it to effectively tackle both common and complex long-tailed scenarios. Our proposed PlanAgent is evaluated on the large-scale and challenging nuPlan benchmarks. A comprehensive set of experiments convincingly demonstrates that PlanAgent outperforms the existing state-of-the-art in the closed-loop motion planning task.
- 12 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. PP (23798920) , 1-14
Top Cited PapersPlanAgent: A Multi-modal Large Language Agent for Closed-loop Vehicle Motion Planning
Vehicle motion planning is an essential component of autonomous driving technology. Current rule-based vehicle motion planning methods perform satisfactorily in common scenarios but struggle to generalize to long-tailed situations. Meanwhile, learning-based methods have yet to achieve superior performance over rule-based approaches in large-scale closed-loop scenarios. To address these issues, we propose PlanAgent, the first mid-to-mid planning system based on a multi-modal large language model (MLLM). MLLM is used as a cognitive agent to introduce human-like knowledge, interpretability, and commonsense reasoning into closed-loop planning. Specifically, PlanAgent leverages the power of MLLM through three core modules. First, an Environment Transformation module constructs a Bird’s Eye View (BEV) map and a lane-graph-based textual description from the environment as inputs. Second, a Reasoning Engine module introduces a hierarchical chain-of-thought from scene understanding to lateral and longitudinal motion instructions, culminating in planner code generation. Last, a Reflection module is integrated to simulate and evaluate the generated planner for reducing MLLM’s uncertainty. PlanAgent is endowed with the common-sense reasoning and generalization capability of MLLM, which empowers it to effectively tackle both common and complex long-tailed scenarios. Our proposed PlanAgent is evaluated on the large-scale and challenging nuPlan benchmarks. A comprehensive set of experiments convincingly demonstrates that PlanAgent outperforms the existing state-of-the-art in the closed-loop motion planning task.
- 19 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (3) , 3566-3573
OccTENS: 3D Occ upancy World Model via Te mporal N ext- S cale Prediction
In this paper, we propose OccTENS, a generative occupancy world model that enables controllable, high-fidelity long-term occupancy generation while maintaining computational efficiency. Different from visual generation, the occupancy world model must capture the fine-grained 3D geometry and dynamic evolution of the 3D scenes, posing great challenges for the generative models. Recent approaches based on autoregression (AR) have demonstrated the potential to predict vehicle movement and future occupancy scenes simultaneously from historical observations, but they typically suffer from inefficiency, temporal degradation in long-term generation and lack of controllability. To holistically address these issues, we reformulate the occupancy world model as a temporal next-scale prediction (TENS) task, which decomposes the temporal sequence modeling problem into the modeling of spatial scale-by-scale generation and temporal scene-by-scene prediction. With a TensFormer, OccTENS can effectively manage the temporal causality and spatial relationships of occupancy sequences in a flexible and scalable way. To enhance the pose controllability, we further propose a holistic pose aggregation strategy, which features a unified sequence modeling for occupancy and ego-motion. Experiments show that OccTENS outperforms the state-of-the-art method with both higher occupancy quality and faster inference time.
- 19 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (3) , 3566-3573
OccTENS: 3D Occ upancy World Model via Te mporal N ext- S cale Prediction
In this paper, we propose OccTENS, a generative occupancy world model that enables controllable, high-fidelity long-term occupancy generation while maintaining computational efficiency. Different from visual generation, the occupancy world model must capture the fine-grained 3D geometry and dynamic evolution of the 3D scenes, posing great challenges for the generative models. Recent approaches based on autoregression (AR) have demonstrated the potential to predict vehicle movement and future occupancy scenes simultaneously from historical observations, but they typically suffer from inefficiency, temporal degradation in long-term generation and lack of controllability. To holistically address these issues, we reformulate the occupancy world model as a temporal next-scale prediction (TENS) task, which decomposes the temporal sequence modeling problem into the modeling of spatial scale-by-scale generation and temporal scene-by-scene prediction. With a TensFormer, OccTENS can effectively manage the temporal causality and spatial relationships of occupancy sequences in a flexible and scalable way. To enhance the pose controllability, we further propose a holistic pose aggregation strategy, which features a unified sequence modeling for occupancy and ego-motion. Experiments show that OccTENS outperforms the state-of-the-art method with both higher occupancy quality and faster inference time.
- 12 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (2) , 1738-1745
TakeAD: Preference-Based Post-Optimization for End-to-End Autonomous Driving With Expert Takeover Data
Existing end-to-end autonomous driving methods typically rely on imitation learning (IL) but face a key challenge: the misalignment between open-loop training and closed-loop deployment. This misalignment often triggers driver-initiated takeovers and system disengagements during closed-loop execution. How to leverage those expert takeover data from disengagement scenarios and effectively expand the IL policy's capability presents a valuable yet unexplored challenge. In this letter, we propose TakeAD, a novel preference-based post-optimization framework that fine-tunes the pre-trained IL policy with this disengagement data to enhance the closed-loop driving performance. First, we design an efficient expert takeover data collection pipeline inspired by human takeover mechanisms in real-world autonomous driving systems. Then, this post-optimization framework integrates iterative Dataset Aggregation (DAgger) for imitation learning with Direct Preference Optimization (DPO) for preference alignment. The DAgger stage equips the policy with fundamental capabilities to handle disengagement states through direct imitation of expert interventions. Subsequently, the DPO stage refines the policy's behavior to better align with expert preferences in disengagement scenarios. Through multiple iterations, the policy progressively learns recovery strategies for disengagement states, thereby mitigating the open-loop gap. Experiments on the closed-loop Bench2Drive benchmark demonstrate our method's effectiveness compared with pure IL methods, with comprehensive ablations confirming the contribution of each component.
- 12 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (2) , 1738-1745
TakeAD: Preference-Based Post-Optimization for End-to-End Autonomous Driving With Expert Takeover Data
Existing end-to-end autonomous driving methods typically rely on imitation learning (IL) but face a key challenge: the misalignment between open-loop training and closed-loop deployment. This misalignment often triggers driver-initiated takeovers and system disengagements during closed-loop execution. How to leverage those expert takeover data from disengagement scenarios and effectively expand the IL policy's capability presents a valuable yet unexplored challenge. In this letter, we propose TakeAD, a novel preference-based post-optimization framework that fine-tunes the pre-trained IL policy with this disengagement data to enhance the closed-loop driving performance. First, we design an efficient expert takeover data collection pipeline inspired by human takeover mechanisms in real-world autonomous driving systems. Then, this post-optimization framework integrates iterative Dataset Aggregation (DAgger) for imitation learning with Direct Preference Optimization (DPO) for preference alignment. The DAgger stage equips the policy with fundamental capabilities to handle disengagement states through direct imitation of expert interventions. Subsequently, the DPO stage refines the policy's behavior to better align with expert preferences in disengagement scenarios. Through multiple iterations, the policy progressively learns recovery strategies for disengagement states, thereby mitigating the open-loop gap. Experiments on the closed-loop Bench2Drive benchmark demonstrate our method's effectiveness compared with pure IL methods, with comprehensive ablations confirming the contribution of each component.
- 8 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (2) , 2178-2185
Mimir: Hierarchical Goal-Driven Diffusion With Uncertainty Propagation for End-to-End Autonomous Driving
End-to-end autonomous driving has emerged as a pivotal direction in the field of autonomous systems. Recent works have demonstrated impressive performance by incorporating high-level guidance signals to steer low-level trajectory planners. However, their potential is often constrained by inaccurate high-level guidance and the computational overhead of complex guidance modules. To address these limitations, we propose Mimir, a novel hierarchical dual-system framework capable of generating robust trajectories relying on goal points with uncertainty estimation: (1) Unlike previous approaches that deterministically model, we estimate goal point uncertainty with a Laplace distribution to enhance robustness; (2) To overcome the slow inference speed of the guidance system, we introduce a multi-rate guidance mechanism that predicts extended goal points in advance. Validated on challenging Navhard and Navtest benchmarks, Mimir surpasses previous state-of-the-art methods with a 20% improvement in the driving score EPDMS, while achieving 1.6× improvement in high-level module inference speed without compromising accuracy. The code and models will be released soon to promote reproducibility and further development.
- 8 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (2) , 2178-2185
Mimir: Hierarchical Goal-Driven Diffusion With Uncertainty Propagation for End-to-End Autonomous Driving
End-to-end autonomous driving has emerged as a pivotal direction in the field of autonomous systems. Recent works have demonstrated impressive performance by incorporating high-level guidance signals to steer low-level trajectory planners. However, their potential is often constrained by inaccurate high-level guidance and the computational overhead of complex guidance modules. To address these limitations, we propose Mimir, a novel hierarchical dual-system framework capable of generating robust trajectories relying on goal points with uncertainty estimation: (1) Unlike previous approaches that deterministically model, we estimate goal point uncertainty with a Laplace distribution to enhance robustness; (2) To overcome the slow inference speed of the guidance system, we introduce a multi-rate guidance mechanism that predicts extended goal points in advance. Validated on challenging Navhard and Navtest benchmarks, Mimir surpasses previous state-of-the-art methods with a 20% improvement in the driving score EPDMS, while achieving 1.6× improvement in high-level module inference speed without compromising accuracy. The code and models will be released soon to promote reproducibility and further development.
- Open Access
- 26 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 31-39
Learning Language-guided Robotic Manipulation via Efficient and Generalizable Feature Distillation
- 26 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 31-39
Open AccessLearning Language-guided Robotic Manipulation via Efficient and Generalizable Feature Distillation