What You See is What You Get: Exploiting Visibility for 3D Object Detection
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 10998-11006
Abstract
Recent advances in 3D sensing have created unique challenges for computer vision. One fundamental challenge is finding a good representation for 3D sensor data. Most popular representations (such as PointNet) are proposed in the context of processing truly 3D data (e.g. points sampled from mesh models), ignoring the fact that 3D sensored data such as a LiDAR sweep is in fact 2.5D. We argue that representing 2.5D data as collections of (x,y,z) points fundamentally destroys hidden information about freespace. In this paper, we demonstrate such knowledge can be efficiently recovered through 3D raycasting and readily incorporated into batch-based gradient learning. We describe a simple approach to augmenting voxel-based networks with visibility: we add a voxelized visibility map as an additional input stream. In addition, we show that visibility can be combined with two crucial modifications common to state-of-the-art 3D detectors: synthetic data augmentation of virtual objects and temporal aggregation of LiDAR sweeps over multiple time frames. On the NuScenes 3D detection benchmark, we show that, by adding an additional stream for visibility input, we can significantly improve the overall detection accuracy of a state-of-the-art 3D detector.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- SECOND: Sparsely Embedded Convolutional DetectionSensors, 2018
- Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net2018
- Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models2017
- Geometric Deep Learning: Going beyond Euclidean dataIEEE Signal Processing Magazine, 2017
- Multimodal deep learning for robust RGB-D object recognition2015
- Vision meets robotics: The KITTI datasetThe International Journal of Robotics Research, 2013
- OctoMap: an efficient probabilistic 3D mapping framework based on octreesAutonomous Robots, 2013
- Fast 3D recognition and pose using the Viewpoint Feature Histogram2010
- A probabilistic representation of LiDAR range data for efficient 3D object detection2008
- Using spin images for efficient object recognition in cluttered 3D scenesIEEE Transactions on Pattern Analysis and Machine Intelligence, 1999