Zhongyu Rao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Zhongyu Rao has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Hai Wang
Jiangsu University
2 shared publications - Long Chen
Jiangsu University
2 shared publications - Yicheng Li
Automotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013, China
2 shared publications - Yingfeng Cai
Jiangsu University
2 shared publications Qingchao Liu
Automotive Engineering Research Institute, Jiangsu University, Zhenjiang, China
1 shared publication- Yilin Zhong
BYD Auto Industry Company Ltd., Shenzhen, China
1 shared publication
Latest Publications
- 17 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Transportation Electrification
- Vol. 11 (1) , 4443-4455
A Camera-Based End-to-End Autonomous Driving Framework Combined With Meta-Based Multi-Task Optimization
Most existing autonomous driving pipelines can be divided into two broad categories: those based on a modular framework, which can result in error transmission, and those based on an end-to-end framework, which lack interpretability. To overcome these challenges, we propose a novel vision-based multi-task framework that incorporates motion planning, Bird’s Eye View(BEV) map generation, BEV object prediction, depth estimation, semantic segmentation, and velocity prediction. In particular, we present an improved view transformation module that transforms feature maps into BEV space and predicts future waypoints in BEV space. The multi-task framework can improve performance by sharing information across tasks, and the results of the multi-tasks also improve interpretability. In addition, to address the negative transfer, we introduce inter-task affinity, which provides a rough estimate of the relationship between tasks. Moreover, because these relationships may change during training, we use a meta-based multi-task optimization method to dynamically adjust the multi-task weighting. We evaluate the performance of our proposed model using the Longest6 and Town05 Long benchmarks of the CARLA simulator. Our model outperforms the current state-of-the-art camera-based models and achieves competitive results with other multi-modal methods on both benchmarks. These results demonstrate the considerable potential of our proposed model for autonomous driving systems. we have also prepared an autonomous driving demonstration using the CARLA simulator which is presented at https://www.youtube.com/watch?v=ctngFH4GSBc.
- 17 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Transportation Electrification
- Vol. 11 (1) , 4443-4455
A Camera-Based End-to-End Autonomous Driving Framework Combined With Meta-Based Multi-Task Optimization
Most existing autonomous driving pipelines can be divided into two broad categories: those based on a modular framework, which can result in error transmission, and those based on an end-to-end framework, which lack interpretability. To overcome these challenges, we propose a novel vision-based multi-task framework that incorporates motion planning, Bird’s Eye View(BEV) map generation, BEV object prediction, depth estimation, semantic segmentation, and velocity prediction. In particular, we present an improved view transformation module that transforms feature maps into BEV space and predicts future waypoints in BEV space. The multi-task framework can improve performance by sharing information across tasks, and the results of the multi-tasks also improve interpretability. In addition, to address the negative transfer, we introduce inter-task affinity, which provides a rough estimate of the relationship between tasks. Moreover, because these relationships may change during training, we use a meta-based multi-task optimization method to dynamically adjust the multi-task weighting. We evaluate the performance of our proposed model using the Longest6 and Town05 Long benchmarks of the CARLA simulator. Our model outperforms the current state-of-the-art camera-based models and achieves competitive results with other multi-modal methods on both benchmarks. These results demonstrate the considerable potential of our proposed model for autonomous driving systems. we have also prepared an autonomous driving demonstration using the CARLA simulator which is presented at https://www.youtube.com/watch?v=ctngFH4GSBc.
- 15 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 9 (10) , 6326-6339
Enhancing Autonomous Driving: A Low-Cost Monocular End-to-End Framework With Multi-Task Integration and Temporal Fusion
End to end autonomous driving system has rapidly progressed and garnered significant attention. Recently, multimodal fusion methods have surpassed vision-based methods in terms of performance. However, the multi-modal methods are both costly and time-confusion. This paper aims to propose a low-cost monocular end-to-end multi-task autonomous driving framework to address these challenges. In particular, to improve the performance of the vision-based model, our model considers three key aspects. First, to enhance the level of detail and visual representation of the surrounding environment, depth embeddings and semantic embeddings are utilized to assist in the perception of the Bird's Eye View(BEV) space. In addition, the motion queries are separated from the BEV queries and the novel transformer-based motion decoder is proposed to generate the output waypoints and target speed for motion planning. Subsequently, to optimize the utilization of the historic features, we introduced a streaming-based processing temporal module to fuse current and historic features. The performance of our proposed model is evaluated using the longest-06 and Town05-long benchmarks within the CARLA-0.9.10.1 simulator. The experiment results demonstrate that our proposed model performs state-of-the-art results on the longest-06 benchmark and promising results on the Town05-long benchmark even compared to these multi-modal sensor based models.
- 15 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 9 (10) , 6326-6339
Enhancing Autonomous Driving: A Low-Cost Monocular End-to-End Framework With Multi-Task Integration and Temporal Fusion
End to end autonomous driving system has rapidly progressed and garnered significant attention. Recently, multimodal fusion methods have surpassed vision-based methods in terms of performance. However, the multi-modal methods are both costly and time-confusion. This paper aims to propose a low-cost monocular end-to-end multi-task autonomous driving framework to address these challenges. In particular, to improve the performance of the vision-based model, our model considers three key aspects. First, to enhance the level of detail and visual representation of the surrounding environment, depth embeddings and semantic embeddings are utilized to assist in the perception of the Bird's Eye View(BEV) space. In addition, the motion queries are separated from the BEV queries and the novel transformer-based motion decoder is proposed to generate the output waypoints and target speed for motion planning. Subsequently, to optimize the utilization of the historic features, we introduced a streaming-based processing temporal module to fuse current and historic features. The performance of our proposed model is evaluated using the longest-06 and Town05-long benchmarks within the CARLA-0.9.10.1 simulator. The experiment results demonstrate that our proposed model performs state-of-the-art results on the longest-06 benchmark and promising results on the Town05-long benchmark even compared to these multi-modal sensor based models.