PUPS: Point Cloud Unified Panoptic Segmentation
Open Access
- 26 June 2023
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 37 (2) , 2339-2347
Abstract
Point cloud panoptic segmentation is a challenging task that seeks a holistic solution for both semantic and instance segmentation to predict groupings of coherent points. Previous approaches treat semantic and instance segmentation as surrogate tasks, and they either use clustering methods or bounding boxes to gather instance groupings with costly computation and hand-craft designs in the instance segmentation task. In this paper, we propose a simple but effective point cloud unified panoptic segmentation (PUPS) framework, which use a set of point-level classifiers to directly predict semantic and instance groupings in an end-to-end manner. To realize PUPS, we introduce bipartite matching to our training pipeline so that our classifiers are able to exclusively predict groupings of instances, getting rid of hand-crafted designs, e.g. anchors and Non-Maximum Suppression (NMS). In order to achieve better grouping results, we utilize a transformer decoder to iteratively refine the point classifiers and develop a context-aware CutMix augmentation to overcome the class imbalance problem. As a result, PUPS achieves 1st place on the leader board of SemanticKITTI panoptic segmentation task and state-of-the-art results on nuScenes.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation2023
- LiDAR-based Panoptic Segmentation via Dynamic Shifting Network2021
- Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation2021
- End-to-End Object Detection with Transformers2020
- nuScenes: A Multimodal Dataset for Autonomous Driving2020
- Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation2020
- Panoptic Feature Pyramid Networks2019
- PointPillars: Fast Encoders for Object Detection From Point Clouds2019
- Seamless Scene Segmentation2019
- Focal Loss for Dense Object Detection2017