Efficient 3D Semantic Segmentation with Superpoint Transformer
Authors:
- 1 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 17149-17158
Abstract
We introduce a novel superpoint-based transformer architecture for efficient semantic segmentation of large-scale 3D scenes. Our method incorporates a fast algorithm to partition point clouds into a hierarchical superpoint structure, which makes our preprocessing 7 times faster than existing superpoint-based approaches. Additionally, we leverage a self-attention mechanism to capture the relationships between superpoints at multiple scales, leading to state-of-the-art performance on three challenging benchmark datasets: S3DIS (76.0% mIoU 6-fold validation), KITTI-360 (63.5% on Val), and DALES (79.6%). With only 212k parameters, our approach is up to 200 times more compact than other state-of-the-art models while maintaining similar performance. Furthermore, our model can be trained on a single GPU in 3 hours for a fold of the S3DIS dataset, which is 7× to 70× fewer GPU-hours than the best-performing methods. Our code and models are accessible at github.com/drprojects/superpoint_transformer.Keywords
All Related Versions
This publication has 44 references indexed in Scilit:
- Hierarchical-Based Semantic Segmentation of 3D Point Cloud Using Deep Learning2023
- Online Segmentation of LiDAR Sequences: Dataset and Algorithm2022
- Hierarchical semantic segmentation of urban scene point clouds via group proposal and graph attention networkInternational Journal of Applied Earth Observation and Geoinformation, 2021
- PCT: Point cloud transformerComputational Visual Media, 2021
- Torch-Points3D: A Modular Multi-Task Framework for Reproducible Deep Learning on 3D Point Clouds2020
- ConvPoint: Continuous convolutions for point cloud processingComputers & Graphics, 2020
- Toward better boundary preserved supervoxel segmentation for 3D point cloudsISPRS Journal of Photogrammetry and Remote Sensing, 2018
- Hierarchical image simplification and segmentation based on Mumford–Shah-salient level line selectionPattern Recognition Letters, 2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Point cloud compression based on hierarchical point clustering2013