Toward Effective 3D Object Detection via Multimodal Fusion to Automatic Driving for Industrial Cyber-Physical Systems
- 12 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Cyber-Physical Systems
- Vol. 2, 281-291
Abstract
AI-empowered automatic driving has experienced rapid development in industrial cyber-physical systems (CPSs), especially in safety vehicles and driverless technologies. 3D object detection is an important task for perceiving the surrounding environment and supporting decision-making when vehicles are on the road, and is also a focus in CPSs. Light detection and ranging (LiDAR)-based detection methods usually lack semantic information, resulting in high uncertainty with incorrect outputs. Thus, handling complex road scenes is difficult. Some data fusion-based methods have been developed to solve these issues. However, the spatiotemporal data misalignment between different sensors is prone to losing information during data fusion. This paper proposes exploiting multimodal information to learn more high-level features to address these issues thus reducing the uncertainty of 3D object detection. First, the VxMLA (voxel and multilevel attention) framework is employed to improve point cloud identification and modeling during 3D object detection. Second, the MF-CAMRL (modal fusion-based channel attention and multidimensional regression loss) model is proposed with two subnetworks. Our model encompasses two strategies, i.e., a multimodal fusion and a deep learning model based on CAMRL. One focuses on the semantic complementarity and geometric proximity for decisionlevel fusion. The other focuses on weighted ensemble bounding boxes to fully utilize the highlevel decision information derived from both modalities and reduce the information loss incurred during modal fusion. Finally, sufficient experiments are performed on the KITTI dataset and presented. The results show that our method is superior to baseline methods.Keywords
Funding Information
- National Key Research and Development Program of China (2022YFF0902500)
- National Natural Science Foundation of China (92367103)
- Open Foundation of Yunnan Key Laboratory of Software Engineering (2023SE301)
This publication has 37 references indexed in Scilit:
- 3D Object Detection for Autonomous Driving: A SurveyPattern Recognition, 2022
- Spatial information enhancement network for 3D object detection from point cloudPattern Recognition, 2022
- Weighted boxes fusion: Ensembling boxes from different object detection modelsImage and Vision Computing, 2021
- A Survey of Autonomous Driving: Common Practices and Emerging TechnologiesIEEE Access, 2020
- A review: Deep learning for medical image segmentation using multi-modality fusionArray, 2019
- Feature Pyramid Networks for Object Detection2017
- Multi-view 3D Object Detection Network for Autonomous Driving2017
- Deep MANTA: A Coarse-to-Fine Many-Task Network for Joint 2D and 3D Vehicle Analysis from Monocular Image2017
- 3D Bounding Box Estimation Using Deep Learning and Geometry2017
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012