Fusing Bird View LIDAR Point Cloud and Front View Camera Image for Deep Object Detection
Preprint
Abstract
We propose a new method for fusing a LIDAR point cloud and camera-captured images in the deep convolutional neural network (CNN). The proposed method constructs a new layer called non-homogeneous pooling layer to transform features between bird view map and front view map. The sparse LIDAR point cloud is used to construct the mapping between the two maps. The pooling layer allows efficient fusion of the bird view and front view features at any stage of the network. This is favorable for the 3D-object detection using camera-LIDAR fusion in autonomous driving scenarios. A corresponding deep CNN is designed and tested on the KITTI bird view object detection dataset, which produces 3D bounding boxes from the bird view map. The fusion method shows particular benefit for detection of pedestrians in the bird view compared to other fusion-based object detection networks.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Focal Loss for Dense Object DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud2018
- Feature Pyramid Networks for Object Detection2017
- YOLO9000: Better, Faster, Stronger2017
- Multi-view 3D Object Detection Network for Autonomous Driving2017
- Deep MANTA: A Coarse-to-Fine Many-Task Network for Joint 2D and 3D Vehicle Analysis from Monocular Image2017
- 3D Bounding Box Estimation Using Deep Learning and Geometry2017
- Accurate Single Stage Detector Using Recurrent Rolling Convolution2017
- Vehicle Detection from 3D Lidar Using Fully Convolutional Network2016