Learning Depth-Guided Convolutions for Monocular 3D Object Detection
Preprint
Abstract
3D object detection from a single image without LiDAR is a challenging task due to the lack of accurate depth information. Conventional 2D convolutions are unsuitable for this task because they fail to capture local object and its scale information, which are vital for 3D object detection. To better represent 3D structure, prior arts typically transform depth maps estimated from 2D images into a pseudo-LiDAR representation, and then apply existing 3D point-cloud based object detectors. However, their results depend heavily on the accuracy of the estimated depth maps, resulting in suboptimal performance. In this work, instead of using pseudo-LiDAR representation, we improve the fundamental 2D fully convolutions by proposing a new local convolutional network (LCN), termed Depth-guided Dynamic-Depthwise-Dilated LCN (D$^4$LCN), where the filters and their receptive fields can be automatically learned from image-based depth maps, making different pixels of different images have different filters. D$^4$LCN overcomes the limitation of conventional 2D convolutions and narrows the gap between image representation and 3D point cloud representation. Extensive experiments show that D$^4$LCN outperforms existing works by large margins. For example, the relative improvement of D$^4$LCN against the state-of-the-art on KITTI is 9.1\% in the moderate setting. The code is available at https://github.com/dingmyu/D4LCN.Keywords
All Related Versions
This publication has 39 references indexed in Scilit:
- Dynamic Filtering with Large Sampling Field for ConvNets2018
- Deep Continuous Fusion for Multi-sensor 3D Object Detection2018
- Deep MANTA: A Coarse-to-Fine Many-Task Network for Joint 2D and 3D Vehicle Analysis from Monocular Image2017
- Deep Residual Learning for Image Recognition2016
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation2016
- Monocular 3D Object Detection for Autonomous Driving2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- Data-driven 3D Voxel Patterns for object category recognition2015
- Are Cars Just 3D Boxes? Jointly Estimating the 3D Shape of Multiple Objects2014
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012