LiDAR-Camera Fusion in Perspective View for 3D Object Detection in Surface Mine

Authors:
Publisher website
Google Scholar
Abstract
LiDAR-Camera fusion can effectively provide the complementary geometric and appearance information for 3D object detection task of autonomous driving system. However, current dominant methods designed for urban scenes often fuse multi-modal information into a Bird's-Eye-View (BEV) feature representation and have a relatively short perception range, which limits their application in surface mine. To achieve long-range perception in surface mine, we utilize a solid-state LiDAR (i.e. Livox Horizon) that has a large perception range (over 200 meters). Accordingly, we propose PVFusion, a novel 3D object detection method that only fuses LiDAR and camera information in Perspective View (PV). Specifically, we first generate the multi-channel depth map via point cloud projection and Depth Map Densification (DMD). Then, we extract visual-depth features and local geometry features from depth-enhanced camera image and point cloud respectively. To better fuse multi-modal information, we propose an Attentional Feature Fusion (AFF) module to dynamically fuse visual-depth features and local geometry features into fused PV features. Moreover, we design a three-branch detection head that leverages a Depth Decoupling Strategy (DDS) to improve the depth estimation for distant objects. We conduct extensive experiments on our surface mine dataset and verify the effectiveness of the proposed PVFusion.
Funding Information
  • National Key Research and Development Program of China Project 3 (2022YFB4703703)