SEGCloud: Semantic Segmentation of 3D Point Clouds
Preprint
Abstract
3D semantic scene labeling is fundamental to agents operating in the real world. In particular, labeling raw 3D point sets from sensors provides fine-grained semantics. Recent works leverage the capabilities of Neural Networks (NNs), but are limited to coarse voxel predictions and do not explicitly enforce global consistency. We present SEGCloud, an end-to-end framework to obtain 3D point-level segmentation that combines the advantages of NNs, trilinear interpolation(TI) and fully connected Conditional Random Fields (FC-CRF). Coarse voxel predictions from a 3D Fully Convolutional NN are transferred back to the raw 3D points via trilinear interpolation. Then the FC-CRF enforces global consistency and provides fine-grained semantics on the points. We implement the latter as a differentiable Recurrent NN to allow joint optimization. We evaluate the framework on two indoor and two outdoor 3D datasets (NYU V2, S3DIS, KITTI, Semantic3D.net), and show performance comparable or superior to the state-of-the-art on all datasets.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Deep Projective 3D Semantic Segmentation2017
- Multiview RGB-D Dataset for Object Instance Detection2016
- Deep Residual Learning for Image Recognition2016
- Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images2016
- 3D Semantic Parsing of Large-Scale Indoor Spaces2016
- Learning Where to Classify in Multi-view Semantic Segmentation2014
- Multi-modal Unsupervised Feature Learning for RGB-D Scene Labeling2014
- Natural terrain classification using three‐dimensional ladar data for ground robot mobilityJournal of Field Robotics, 2006
- Efficient Graph-Based Image SegmentationInternational Journal of Computer Vision, 2004
- A chronology of interpolation: from ancient astronomy to modern signal and image processingProceedings of the IEEE, 2002