Mingyu Zhou
Mingyu Zhou has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
- Guirong Zhuo
School of Automotive Studies, Tongji University, Shanghai, China
2 shared publications - Lianqing Zheng
Tongji University
2 shared publications - Lu Xiong
Tongji University
2 shared publications - Shouyi Lu
School of Automotive Studies, Tongji University, Shanghai, China
2 shared publications - Huanyu Zhouins
School of Automotive Studies, Tongji University, Shanghai, China
1 shared publication - Jie Bai
School of Information and Electrical Engineering, Hangzhou City University, Hangzhou, China
1 shared publication
Latest Publications
- 8 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 9 (6) , 5065-5079
4DRVO-Net: Deep 4D Radar–Visual Odometry Using Multi-Modal and Multi-Scale Adaptive Fusion
Four-dimensional (4D) radar–visual odometry (4DRVO) integrates complementary information from 4D radar and cameras, making it an attractive solution for achieving accurate and robust pose estimation. However, 4DRVO may exhibit significant tracking errors owing to three main factors: 1) sparsity of 4D radar point clouds; 2) inaccurate data association and insufficient feature interaction between the 4D radar and camera; and 3) disturbances caused by dynamic objects in the environment, affecting odometry estimation. In this paper, we present 4DRVO-Net, which is a method for 4D radar–visual odometry. This method leverages the feature pyramid, pose warping, and cost volume (PWC) network architecture to progressively estimate and refine poses. Specifically, we propose a multi-scale feature extraction network called Radar-PointNet++ that fully considers rich 4D radar point information, enabling fine-grained learning for sparse 4D radar point clouds. To effectively integrate the two modalities, we design an adaptive 4D radar–camera fusion module (A-RCFM) that automatically selects image features based on 4D radar point features, facilitating multi-scale cross-modal feature interaction and adaptive multi-modal feature fusion. In addition, we introduce a velocity-guided point-confidence estimation module to measure local motion patterns, reduce the influence of dynamic objects and outliers, and provide continuous updates during pose refinement. We demonstrate the excellent performance of our method and the effectiveness of each module design on both the VoD and in-house datasets. Our method outperforms all learning-based and geometry-based methods for most sequences in the VoD dataset. Furthermore, it has exhibited promising performance that closely approaches that of the 64-beam LiDAR odometry results of A-LOAM without mapping optimization.
- 8 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 9 (6) , 5065-5079
4DRVO-Net: Deep 4D Radar–Visual Odometry Using Multi-Modal and Multi-Scale Adaptive Fusion
Four-dimensional (4D) radar–visual odometry (4DRVO) integrates complementary information from 4D radar and cameras, making it an attractive solution for achieving accurate and robust pose estimation. However, 4DRVO may exhibit significant tracking errors owing to three main factors: 1) sparsity of 4D radar point clouds; 2) inaccurate data association and insufficient feature interaction between the 4D radar and camera; and 3) disturbances caused by dynamic objects in the environment, affecting odometry estimation. In this paper, we present 4DRVO-Net, which is a method for 4D radar–visual odometry. This method leverages the feature pyramid, pose warping, and cost volume (PWC) network architecture to progressively estimate and refine poses. Specifically, we propose a multi-scale feature extraction network called Radar-PointNet++ that fully considers rich 4D radar point information, enabling fine-grained learning for sparse 4D radar point clouds. To effectively integrate the two modalities, we design an adaptive 4D radar–camera fusion module (A-RCFM) that automatically selects image features based on 4D radar point features, facilitating multi-scale cross-modal feature interaction and adaptive multi-modal feature fusion. In addition, we introduce a velocity-guided point-confidence estimation module to measure local motion patterns, reduce the influence of dynamic objects and outliers, and provide continuous updates during pose refinement. We demonstrate the excellent performance of our method and the effectiveness of each module design on both the VoD and in-house datasets. Our method outperforms all learning-based and geometry-based methods for most sequences in the VoD dataset. Furthermore, it has exhibited promising performance that closely approaches that of the 64-beam LiDAR odometry results of A-LOAM without mapping optimization.
- 1 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 9 (1) , 879-892
Efficient Deep-Learning 4D Automotive Radar Odometry Method
Odometry is a crucial technology for the autonomous positioning of intelligent vehicles. While estimating the odometry from LiDAR and cameras has progressed recently, it remains to be seen how to estimate the odometry from a 4D radar, an emerging sensor with unique advantages over cameras and LiDAR. In this study, a deep-learning-based 4D radar odometry method, named 4DRO-Net, is proposed. The method employs a coarse-to-fine hierarchical optimization technique based on a sliding window to estimate and refine an autonomous vehicle's pose in an iterative manner. A feature-extraction network for 4D radar point clouds is proposed to achieve efficient learning of sparse point clouds. An initial pose-generation module is constructed to obtain the initial pose, which is used to warp the first point cloud and bridge the distance to the second point cloud. A velocity-aware attention cost volume module is then developed to correlate the warped first point cloud with the second point cloud to obtain point-motion information. The velocity information of the radar points is used to learn the attention weights to increase the robustness of the motion information estimation. Motion information is used to regress the corrected pose, which is then used to refine the initial pose to obtain a more accurate final pose. The superior performance and effectiveness of our 4D radar odometry method are demonstrated on both the View-of-Delft and an in-house dataset.
- 1 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 9 (1) , 879-892
Efficient Deep-Learning 4D Automotive Radar Odometry Method
Odometry is a crucial technology for the autonomous positioning of intelligent vehicles. While estimating the odometry from LiDAR and cameras has progressed recently, it remains to be seen how to estimate the odometry from a 4D radar, an emerging sensor with unique advantages over cameras and LiDAR. In this study, a deep-learning-based 4D radar odometry method, named 4DRO-Net, is proposed. The method employs a coarse-to-fine hierarchical optimization technique based on a sliding window to estimate and refine an autonomous vehicle's pose in an iterative manner. A feature-extraction network for 4D radar point clouds is proposed to achieve efficient learning of sparse point clouds. An initial pose-generation module is constructed to obtain the initial pose, which is used to warp the first point cloud and bridge the distance to the second point cloud. A velocity-aware attention cost volume module is then developed to correlate the warped first point cloud with the second point cloud to obtain point-motion information. The velocity information of the radar points is used to learn the attention weights to increase the robustness of the motion information estimation. Motion information is used to regress the corrected pose, which is then used to refine the initial pose to obtain a more accurate final pose. The superior performance and effectiveness of our 4D radar odometry method are demonstrated on both the View-of-Delft and an in-house dataset.