Multi-Task Cross-Modality Attention-Fusion for 2D Object Detection
Authors:
- 24 September 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3619-3626
Abstract
Accurate and robust object detection is critical for autonomous driving. Image-based detectors face difficulties caused by low visibility in adverse weather conditions. Thus, radar-camera fusion is of particular interest but presents challenges in optimally fusing heterogeneous data sources. To approach this issue, we propose two new radar preprocessing techniques to better align radar and camera data. In addition, we introduce a Multi-Task Cross-Modality Attention-Fusion Network (MCAF-Net) for object detection, which includes two new fusion blocks. These allow for exploiting information from the feature maps more comprehensively. The proposed algorithm jointly detects objects and segments free space, which guides the model to focus on the more relevant part of the scene, namely, the occupied space. Our approach outperforms current state-of-the-art radar-camera fusion-based object detectors in the nuScenes dataset and achieves more robust results in adverse weather conditions and nighttime scenarios.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Radar-Enhanced Image Fusion-based Object Detection for Autonomous Driving2022
- nuScenes: A Multimodal Dataset for Autonomous Driving2020
- RVNet: Deep Sensor Fusion of Monocular Camera and Radar for Image-Based Obstacle Detection in Challenging Environments2019
- Multi-Task Multi-Sensor Fusion for 3D Object Detection2019
- Distant Vehicle Detection Using Radar and Vision2019
- Focal Loss for Dense Object Detection2017
- Mask R-CNN2017
- SSD: Single Shot MultiBox Detector2016
- Monocular 3D Object Detection for Autonomous Driving2016
- Fast R-CNN2015