Vehicle-SceneInteraction: A Text-Driven 3-D Lidar Place Recognition Method for Autonomous Driving
- 26 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. PP (15249050) , 1-13
Abstract
Environment description-based vehicle-scene interaction for vehicle localization in large-scale point cloud maps constructed through multi-sensor systems is critically significant for the advancement of large-scale intelligent transportation systems, such as delivery vehicles operating in the ‘last mile’. However, current vehicle-scene interaction encounter challenges due to the inability of point cloud encoders to effectively capture local details and long-range spatial relationships and a significant modality gap between text and point cloud representations. To address these challenges, we present Des4Pos, a novel two-stage text-driven 3D Lidar place recognition framework. In the coarse stage, the point-cloud encoder utilizes the Multi-scale Fusion Attention Mechanism (MFAM) to enhance local geometric features, followed by a bidirectional Long Short-Term Memory (LSTM) module to strengthen global spatial relationships. Concurrently, the Stepped Text Encoder (STE) integrates cross-modal prior knowledge from CLIP and aligns text and point-cloud features using this prior knowledge, effectively bridging modality discrepancies. In the fine stage, we introduce a Cascaded Residual Attention (CRA) module to fuse cross-modal features and predict relative localization offsets, thereby achieving greater localization precision. Experiments on the KITTI360Pose test set demonstrate that Des4Pos achieves state-of-the-art performance in text-to-point-cloud place recognition at the expense of a slight increase in the runtime. Specifically, it attains a top-1 accuracy of 40% and a top-10 accuracy of 77% under a 5-meter radius threshold, surpassing the best previous sota method Text2Loc by 8% and 7%, respectively. Our code and datasets are publicly available at https://github.com/nuozimiaowu/Des4PosKeywords
All Related Versions
Funding Information
- Natural Science Foundation of Shandong Province (ZR2024QF284)
- National Natural Science Foundation of China (62403364)
- China Postdoctoral Science Foundation (GZC20241214)
- China Postdoctoral Science Foundation (2024M752421)
- Major Innovation Project of Qilu University of Technology (2025ZDZX03)
- Major Innovation Project of Qilu University of Technology (2025ZDZX04)
- Science and Technology Support Plan for Youth Innovation of Colleges and Universities of Shandong Province of China (2023KJ134)
This publication has 46 references indexed in Scilit:
- Place recognition meet multiple modalities: a comprehensive review, current challenges and future developmentArtificial Intelligence Review, 2025
- PointLLM: Empowering Large Language Models to Understand Point Clouds2024
- SegPoint: Segment Any Point Cloud via Large Language Model2024
- Towards Cross-Modal Point Cloud Retrieval for Indoor Scenes2024
- A Survey on Deep Visual Place RecognitionIEEE Access, 2021
- ScanRefer: 3D Object Localization in RGB-D Scans Using Natural Language2020
- ReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes2020
- DH3D: Deep Hierarchical 3D Descriptors for Robust Large-Scale 6DoF Relocalization2020
- PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation2017
- NetVLAD: CNN Architecture for Weakly Supervised Place Recognition2016