Depth-Vision-Decoupled Transformer With Cascaded Group Convolutional Attention for Monocular 3-D Object Detection
- 3 October 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 73 (00189456) , 1-13
Abstract
Monocular three-dimensional (3D) object detection is a challenging and important task in autonomous driving. In 3D object detection, depth information is crucial for determining the position, size, and posture of objects. However, in the two-dimensional (2D) images obtained by monocular cameras, depth information is compressed into a plane, which blurs the relative position and size relationships between objects. In addition, the pose changes of objects in 3D space (such as rotation, tilt, etc.) can cause changes in their projected shape, size, and position in 2D images, which increases the difficulty of detecting 3D objects from monocular 2D images. In this paper, we propose a depth-vision-decoupled transformer for monocular multi-class 3D object detection. The proposed scheme is constructed with the following novel components: (1) a Cascaded Group-Multi Scale Convolutional Attention (CG-MSCA) with multi-scale-perception and direction-perception capabilities to focus on the local characteristics of the complex scene; (2) a Decoupled-Feature-Aware TRansformer (DFTR) module, which globally decouples depth and visual features, encodes and decodes them separately to avoid the monocularly estimated inaccurate depth interfering with the model’s learning of visual information and alleviate the mismatch between stereo information and geometric shape of the object; (3) a Cross-Attention-Guided Fusion Module (CAFM) to rationalize the fusion of decoupled depth and visual features before prediction. Experiments on the KITTI, DAIR-V2X-V, and DAIR-V2X-I datasets show that our proposed method produces competitive performance compared with Transformer-based methods and other state-of-the-art (SOTA) methods. For the Car Category, our method achieves 17.04% Average Precision of three-dimensional (AP3D) for the moderate setting (the most important setting) of the KITTI dataset with an Intersection over Union (IoU) threshold of 0.7.Keywords
Funding Information
- National Key Research and Development Program of China (2021YFC2201902)
- Tianjin Transportation Science and Technology Development Plan (202234)
This publication has 48 references indexed in Scilit:
- Visual attention networkComputational Visual Media, 2023
- DEVIANT: Depth EquiVarIAnt NeTwork for Monocular 3D Object Detection2022
- BEVFormer: Learning Bird’s-Eye-View Representation from Multi-camera Images via Spatiotemporal Transformers2022
- DID-M3D: Decoupling Instance Depth for Monocular 3D Object Detection2022
- FCCDN: Feature constraint network for VHR image change detectionISPRS Journal of Photogrammetry and Remote Sensing, 2022
- HOTR: End-to-End Human-Object Interaction Detection with Transformers2021
- Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection2021
- Delving into Localization Errors for Monocular 3D Object Detection2021
- Objects are Different: Flexible Monocular 3D Object Detection2021
- Learning Depth-Guided Convolutions for Monocular 3D Object Detection2020