3D Single Object Tracking With Cross-Modal Fusion Conflict Elimination
- 17 March 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (5) , 4826-4833
Abstract
3D single object tracking based on point clouds is a key challenge in robotics and autonomous driving technology. Mainstream methods rely on point clouds for geometric matching or motion estimation between the target template and the search area. However, the lack of texture and the sparsity of incomplete point clouds make it difficult for unimodal trackers to distinguish objects with similar structures. To overcome the limitations of previous methods, this paper proposes a cross-modal fusion conflict elimination tracker (CCETrack). The point clouds collected by LiDAR provide accurate depth and shape information about the surrounding environment, while the camera sensor provides RGB images containing rich semantic and texture information. CCETrack fully leverages both modalities to track 3D objects. Specifically, to address cross-modal conflicts caused by heterogeneous sensors, we propose a global context alignment module that aligns RGB images with point clouds and generates enhanced image features. Then, a sparse feature enhancement module is designed to optimize voxelized point cloud features using the rich image features. In the feature fusion stage, both modalities are converted into BEV features, with the template and search area features fused separately. A self-attention mechanism is employed to establish bidirectional communication between regions. Our method maximizes the use of effective information and achieves state-of-the-art performance on the KITTI and nuScenes datasets through multimodal complementarity.Keywords
Funding Information
- National Natural Science Foundation of China (62473205)
- Liaoning Revitalization Talents Program (XLYC2403128)
- Basic and Applied Basic Research Foundation of Guangdong Province (2024B1515250008)
- National Natural Science Foundation of China (U21A20476)
This publication has 35 references indexed in Scilit:
- 3D Siamese Transformer Network for Single Object Tracking on Point Clouds2022
- High speed long-term visual object tracking algorithm for real robot systemsNeurocomputing, 2021
- P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds2020
- nuScenes: A Multimodal Dataset for Autonomous Driving2020
- Leveraging Shape Completion for 3D Siamese Tracking2019
- DGCNN: A convolutional neural network over large-scale labeled graphsNeural Networks, 2018
- High Performance Visual Tracking with Siamese Region Proposal Network2018
- Fully-Convolutional Siamese Networks for Object Tracking2016
- Online Object Tracking: A Benchmark2013
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012