PRFusion: Toward Effective and Robust Multi-Modal Place Recognition With Image and Point Cloud Fusion
- 3 October 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 25 (12) , 20523-20534
Abstract
Place recognition plays a crucial role in the fields of robotics and computer vision, finding applications in areas such as autonomous driving, mapping, and localization. Place recognition identifies a place using query sensor data and a known database. One of the main challenges is to develop a model that can deliver accurate results while being robust to environmental variations. We propose two multi-modal place recognition models, namely PRFusion and PRFusion++. PRFusion utilizes global fusion with manifold metric attention, enabling effective interaction between features without requiring camera-LiDAR extrinsic calibrations. In contrast, PRFusion++ assumes the availability of extrinsic calibrations and leverages pixel-point correspondences to enhance feature learning on local windows. Additionally, both models incorporate neural diffusion layers, which enable reliable operation even in challenging environments. We verify the state-of-the-art performance of both models on three large-scale benchmarks. Notably, they outperform existing models by a substantial margin of $+$ 3.0 AR@1 on the demanding Boreas dataset. Furthermore, we conduct ablation studies to validate the effectiveness of our proposed methods.Keywords
All Related Versions
Funding Information
- Singapore Ministry of Education Academic Research Fund Tier 2 (MOE-T2EP20220-0002)
This publication has 54 references indexed in Scilit:
- LaMAR: Benchmarking Localization and Mapping for Augmented Reality2022
- GSV-Cities: Toward appropriate supervised visual place recognitionNeurocomputing, 2022
- TransVPR: Transformer-Based Place Recognition with Multi-Level Attention Aggregation2022
- Rethinking Visual Geo-localization for Large-Scale Applications2022
- Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition2021
- SOE-Net: A Self-Attention and Orientation Encoding Network for Point Cloud based Place Recognition2021
- Knowledge Distillation: A SurveyInternational Journal of Computer Vision, 2021
- PCAN: 3D Attention Map Learning Using Contextual Information for Point Cloud Based Retrieval2019
- PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition2018
- Deep Metric Learning Using Triplet Network2015