ImFusion: Boosting Two-Stage 3D Object Detection via Image Candidates
- 24 November 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 31 (10709908) , 241-245
Abstract
Multi-modal fusion methods combine the advantages of both point clouds and RGB images to boost the performance of 3D object detection. Despite the significant progress, we find that existing two-stage multi-modal fusion methods suffer from the 3D proposal missing in the first stage and projected-style feature fusion mechanism. To solve these problems, we propose a two-stage multi-modal feature fusion network, which improves the recall rate of hard targets in the first stage of network with pseudo 3D proposals generated from image candidates. Then, considering the complementary information between similar image foreground features across multiple objects, we design a multi-modal cross-target fusion module to pay more attention to the foreground objects. It enables a 3D proposal can aggregate the semantic features of multiple image candidates belonging to the same category. Finally, these enhanced fused proposals are processed in the second stage to further boost the performance of 3D detector. Experimental results on SUN RGB-D and KITTI datasets show the effectiveness of our proposed method.Keywords
Funding Information
- National Key R&D Program of China (2021ZD0114600)
- National Natural Science Foundation of China (62276260, 61976210, 62176254, 62006230, 62002357, 62206290, 62206283)
This publication has 32 references indexed in Scilit:
- STD: Sparse-to-Dense 3D Object Detector for Point Cloud2019
- PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud2019
- Pseudo-LiDAR From Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving2019
- MVX-Net: Multimodal VoxelNet for 3D Object Detection2019
- Joint 3D Proposal Generation and Object Detection from View Aggregation2018
- Frustum PointNets for 3D Object Detection from RGB-D Data2018
- PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation2018
- Multi-view 3D Object Detection Network for Autonomous Driving2017
- Fast R-CNN2015
- SUN RGB-D: A RGB-D scene understanding benchmark suite2015