Zhongyang Xiao
Research Subjects
Zhongyang Xiao has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
- Diange Yang
Tsinghua University
4 shared publications - Kun Jiang
Tsinghua University
4 shared publications - Tuopu Wen
School of Vehicle and Mobility, Tsinghua University, Beijing, China
3 shared publications - Jin Huang
Tsinghua University
2 shared publications - Jinyu Miao
Tsinghua University
2 shared publications - Yunlong Wang
School of Vehicle and Mobility, Tsinghua University, Beijing, China
2 shared publications
Latest Publications
- 22 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (5) , 4583-4590
A Hybrid Approach for Cross-Modality Pose Estimation Between Image and Point Cloud
Cross-modality pose estimation/localization is a critical challenge for multi-sensor-based perception systems, with applications spanning vehicle localization and online calibrations. In this letter, we introduce a hybrid approach to estimate the camera pose with respect to a point cloud with co-visibility. This approach adopts a coarse-to-fine scheme, utilizing a learning-based pose estimator to initially estimate a coarse pose, followed by an optimization-based method for precise geometry alignment. Initially, we propose a neural network utilizing multi-scale cross-attention to predict the co-visible points between the camera and the point cloud. Simultaneously, a coarse pose is estimated from the tightly-coupled dual-modal features. Initialized by the coarsely estimated pose, an optimization-based approach is employed to minimize the reprojection error of the co-visible point cloud onto the imaging plane. The proposed approach is fully evaluated on the KITTI and nuScenes datasets, and superior experimental results are achieved with the comparison of SOTA methods.
- 22 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (5) , 4583-4590
A Hybrid Approach for Cross-Modality Pose Estimation Between Image and Point Cloud
Cross-modality pose estimation/localization is a critical challenge for multi-sensor-based perception systems, with applications spanning vehicle localization and online calibrations. In this letter, we introduce a hybrid approach to estimate the camera pose with respect to a point cloud with co-visibility. This approach adopts a coarse-to-fine scheme, utilizing a learning-based pose estimator to initially estimate a coarse pose, followed by an optimization-based method for precise geometry alignment. Initially, we propose a neural network utilizing multi-scale cross-attention to predict the co-visible points between the camera and the point cloud. Simultaneously, a coarse pose is estimated from the tightly-coupled dual-modal features. Initialized by the coarsely estimated pose, an optimization-based approach is employed to minimize the reprojection error of the co-visible point cloud onto the imaging plane. The proposed approach is fully evaluated on the KITTI and nuScenes datasets, and superior experimental results are achieved with the comparison of SOTA methods.
- 19 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 10 (4) , 2519-2550
A Survey on Monocular Re-Localization: From the Perspective of Scene Map Representation
Monocular Re-Localization (MRL) is an essential component in autonomous applications, estimating 6 degree-of-freedom ego poses w.r.t. the scene map using monocular images. Over the past few decades, there has been notable advancement in the field of MRL techniques. Several works have achieved remarkable success in terms of both the accuracy and reliability of localization. In MRL, scene maps are represented in diverse forms, and they dictate the functionality and efficacy of MRL methods. However, to the best of our knowledge, existing surveys do not provide systematic reviews about the correlation between MRL solutions and their used scene map representation. This survey fills the gap by comprehensively reviewing MRL methods from such a perspective, promoting further research. 1) We commence by delving into the problem definition of MRL, exploring current challenges, and comparing ours with existing surveys. 2) MRL methods are categorized and reviewed into five classes based on the representation forms of the utilized map, i.e., geo-tagged frames, visual landmarks, point clouds, vectorized semantic maps, and neural network-based maps. 3) To quantitatively and fairly compare MRL methods using different maps, we introduce some public datasets and evaluate the performances of several advanced MRL approaches. The strengths and shortcomings of MRL methods using various maps are analyzed. 4) Finally, we introduce some topics of interest in this field and provide our individual viewpoints. This survey can serve as a valuable resource for MRL, and a summary of this survey is accessible to the community at: github.com/jinyummiao/map-in-mono-reloc
- 19 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Vehicles
- Vol. 10 (4) , 2519-2550
A Survey on Monocular Re-Localization: From the Perspective of Scene Map Representation
Monocular Re-Localization (MRL) is an essential component in autonomous applications, estimating 6 degree-of-freedom ego poses w.r.t. the scene map using monocular images. Over the past few decades, there has been notable advancement in the field of MRL techniques. Several works have achieved remarkable success in terms of both the accuracy and reliability of localization. In MRL, scene maps are represented in diverse forms, and they dictate the functionality and efficacy of MRL methods. However, to the best of our knowledge, existing surveys do not provide systematic reviews about the correlation between MRL solutions and their used scene map representation. This survey fills the gap by comprehensively reviewing MRL methods from such a perspective, promoting further research. 1) We commence by delving into the problem definition of MRL, exploring current challenges, and comparing ours with existing surveys. 2) MRL methods are categorized and reviewed into five classes based on the representation forms of the utilized map, i.e., geo-tagged frames, visual landmarks, point clouds, vectorized semantic maps, and neural network-based maps. 3) To quantitatively and fairly compare MRL methods using different maps, we introduce some public datasets and evaluate the performances of several advanced MRL approaches. The strengths and shortcomings of MRL methods using various maps are analyzed. 4) Finally, we introduce some topics of interest in this field and provide our individual viewpoints. This survey can serve as a valuable resource for MRL, and a summary of this survey is accessible to the community at: github.com/jinyummiao/map-in-mono-reloc
- 29 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (1) , 803-810
Poses as Queries: End-to-End Image-to-LiDAR Map Localization With Transformers
High-precision vehicle localization with commercial setups is a crucial technique for high-level autonomous driving tasks. As a newly emerged approach, monocular localization in LiDAR map achieves promising balance between cost and accuracy, but estimating pose by finding correspondences between such cross-modal sensor data is challenging, thereby damaging the localization accuracy. In this letter, we address the problem by proposing a novel Transformer-based neural network to register 2D images into 3D LiDAR map in an end-to-end manner. We first implicitly represent poses as high-dimensional feature vectors called pose queries and gradually optimize poses by interacting with the retrieved relevant information from cross-modal features using attention mechanism in a proposed POse Estimator Transformer (POET) module. Moreover, we apply a multiple hypotheses aggregation method that estimates the final poses by performing parallel optimization on multiple randomly initialized pose queries to reduce the network uncertainty. Comprehensive analysis and experimental results on public benchmarks conclude that the proposed image-to-LiDAR map localization network could achieve state-of-the-art performances in challenging cross-modal localization tasks.
- 29 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (1) , 803-810
Poses as Queries: End-to-End Image-to-LiDAR Map Localization With Transformers
High-precision vehicle localization with commercial setups is a crucial technique for high-level autonomous driving tasks. As a newly emerged approach, monocular localization in LiDAR map achieves promising balance between cost and accuracy, but estimating pose by finding correspondences between such cross-modal sensor data is challenging, thereby damaging the localization accuracy. In this letter, we address the problem by proposing a novel Transformer-based neural network to register 2D images into 3D LiDAR map in an end-to-end manner. We first implicitly represent poses as high-dimensional feature vectors called pose queries and gradually optimize poses by interacting with the retrieved relevant information from cross-modal features using attention mechanism in a proposed POse Estimator Transformer (POET) module. Moreover, we apply a multiple hypotheses aggregation method that estimates the final poses by performing parallel optimization on multiple randomly initialized pose queries to reduce the network uncertainty. Comprehensive analysis and experimental results on public benchmarks conclude that the proposed image-to-LiDAR map localization network could achieve state-of-the-art performances in challenging cross-modal localization tasks.
- Open Access
Monocular Localization with Vector HD Map (MLVHM): A Low-Cost Method for Commercial IVs
Real-time vehicle localization (i.e., position and orientation estimation in the world coordinate system) with high accuracy is the fundamental function of an intelligent vehicle (IV) system. In the process of commercialization of IVs, many car manufacturers attempt to avoid high-cost sensor systems (e.g., RTK GNSS and LiDAR) in favor of low-cost optical sensors such as cameras. The same cost-saving strategy also gives rise to an increasing number of vehicles equipped with High Definition (HD) maps. Rooted upon these existing technologies, this article presents the concept of Monocular Localization with Vector HD Map (MLVHM), a novel camera-based map-matching method that efficiently aligns semantic-level geometric features in-camera acquired frames against the vector HD map in order to achieve high-precision vehicle absolute localization with minimal cost. The semantic features are delicately chosen for the ease of map vector alignment as well as for the resiliency against occlusion and fluctuation in illumination. The effective data association method in MLVHM serves as the basis for the camera position estimation by minimizing feature re-projection errors, and the frame-to-frame motion fusion is further introduced for reliable localization results. Experiments have shown that MLVHM can achieve high-precision vehicle localization with an RMSE of 24 cm with no cumulative error. In addition, we use low-cost on-board sensors and light-weight HD maps to achieve or even exceed the accuracy of existing map-matching algorithms.
Open AccessMonocular Localization with Vector HD Map (MLVHM): A Low-Cost Method for Commercial IVs
Real-time vehicle localization (i.e., position and orientation estimation in the world coordinate system) with high accuracy is the fundamental function of an intelligent vehicle (IV) system. In the process of commercialization of IVs, many car manufacturers attempt to avoid high-cost sensor systems (e.g., RTK GNSS and LiDAR) in favor of low-cost optical sensors such as cameras. The same cost-saving strategy also gives rise to an increasing number of vehicles equipped with High Definition (HD) maps. Rooted upon these existing technologies, this article presents the concept of Monocular Localization with Vector HD Map (MLVHM), a novel camera-based map-matching method that efficiently aligns semantic-level geometric features in-camera acquired frames against the vector HD map in order to achieve high-precision vehicle absolute localization with minimal cost. The semantic features are delicately chosen for the ease of map vector alignment as well as for the resiliency against occlusion and fluctuation in illumination. The effective data association method in MLVHM serves as the basis for the camera position estimation by minimizing feature re-projection errors, and the frame-to-frame motion fusion is further introduced for reliable localization results. Experiments have shown that MLVHM can achieve high-precision vehicle localization with an RMSE of 24 cm with no cumulative error. In addition, we use low-cost on-board sensors and light-weight HD maps to achieve or even exceed the accuracy of existing map-matching algorithms.
- Open Access
A Unified Multiple-Target Positioning Framework for Intelligent Connected Vehicles
Future intelligent transport systems depend on the accurate positioning of multiple targets in the road scene, including vehicles and all other moving or static elements. The existing self-positioning capability of individual vehicles remains insufficient. Also, bottlenecks in developing on-board perception systems stymie further improvements in the precision and integrity of positioning targets. Vehicle-to-everything (V2X) communication, which is fast becoming a standard component of intelligent and connected vehicles, renders new sources of information such as dynamically updated high-definition (HD) maps accessible. In this paper, we propose a unified theoretical framework for multiple-target positioning by fusing multi-source heterogeneous information from the on-board sensors and V2X technology of vehicles. Numerical and theoretical studies are conducted to evaluate the performance of the framework proposed. With a low-cost global navigation satellite system (GNSS) coupled with an initial navigation system (INS), on-board sensors, and a normally equipped HD map, the precision of multiple-target positioning attained can meet the requirements of high-level automated vehicles. Meanwhile, the integrity of target sensing is significantly improved by the sharing of sensor information and exploitation of map data. Furthermore, our framework is more adaptable to traffic scenarios when compared with state-of-the-art techniques.
Open AccessA Unified Multiple-Target Positioning Framework for Intelligent Connected Vehicles
Future intelligent transport systems depend on the accurate positioning of multiple targets in the road scene, including vehicles and all other moving or static elements. The existing self-positioning capability of individual vehicles remains insufficient. Also, bottlenecks in developing on-board perception systems stymie further improvements in the precision and integrity of positioning targets. Vehicle-to-everything (V2X) communication, which is fast becoming a standard component of intelligent and connected vehicles, renders new sources of information such as dynamically updated high-definition (HD) maps accessible. In this paper, we propose a unified theoretical framework for multiple-target positioning by fusing multi-source heterogeneous information from the on-board sensors and V2X technology of vehicles. Numerical and theoretical studies are conducted to evaluate the performance of the framework proposed. With a low-cost global navigation satellite system (GNSS) coupled with an initial navigation system (INS), on-board sensors, and a normally equipped HD map, the precision of multiple-target positioning attained can meet the requirements of high-level automated vehicles. Meanwhile, the integrity of target sensing is significantly improved by the sharing of sensor information and exploitation of map data. Furthermore, our framework is more adaptable to traffic scenarios when compared with state-of-the-art techniques.