CrossGLoc: Cross-Modal Global Localization Leveraging Pretrained Diffusion Models and Semantic Cues for Intelligent Vehicles
- 31 July 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 26 (11) , 18513-18524
Abstract
Cross-modal global localization matches visual information with pre-built LiDAR maps, which has attracted more and more attention for its low cost and potential robustness. However, the inherent modality difference between images and point clouds makes it challenging. This paper proposes a novel cross-modal global localization system, named CrossGLoc, which leverages pre-trained diffusion models and semantic cues to address this challenge. The main idea is leveraging the semantic cues shared between different modalities to bridge the modality gap, and utilizing pre-trained diffusion models to extract modality-consistent high-dimensional features guided by these semantic cues. To achieve this, ControlNet is used to generate intermediate feature maps from semantic images and semantic map projections, and a semantic categories-based feature aggregation algorithm is proposed to aggregate these feature maps into global descriptors. Furthermore, a semantic edge key points-based pose estimation algorithm is proposed to estimate the pose of retrieved image and point cloud pairs. Extensive experiments on the KITTI dataset, the KITTI360 dataset and the self-collected dataset demonstrate that the proposed method achieves state-of-the-art performance in cross-modal global localization.Keywords
Funding Information
- National Natural Science Foundation of China (U22A20100, 62203294, 62173228)
This publication has 49 references indexed in Scilit:
- I2D-Loc: Camera localization via image to LiDAR depth flowISPRS Journal of Photogrammetry and Remote Sensing, 2022
- OverlapNet: a siamese network for computing LiDAR scan similarity with applications to loop closing and localizationAutonomous Robots, 2021
- PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition2018
- In Defense of Classical Image Processing: Fast Depth Completion on the CPU2018
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- NetVLAD: CNN Architecture for Weakly Supervised Place Recognition2016
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012
- Aggregating local descriptors into a compact image representation2010
- Speeded-Up Robust Features (SURF)Computer Vision and Image Understanding, 2008
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004