Improving Panoptic Segmentation at All Scales
Authors:
- Lorenzo PorziLPLorenzo Porzi
- Samuel Rota BulòSBSamuel Rota Bulò
- Peter KontschiederPKPeter Kontschieder
Preprint
Abstract
Crop-based training strategies decouple training resolution from GPU memory consumption, allowing the use of large-capacity panoptic segmentation networks on multi-megapixel images. Using crops, however, can introduce a bias towards truncating or missing large objects. To address this, we propose a novel crop-aware bounding box regression loss (CABB loss), which promotes predictions to be consistent with the visible parts of the cropped objects, while not over-penalizing them for extending outside of the crop. We further introduce a novel data sampling and augmentation strategy which improves generalization across scales by counteracting the imbalanced distribution of object sizes. Combining these two contributions with a carefully designed, top-down panoptic segmentation architecture, we obtain new state-of-the-art results on the challenging Mapillary Vistas (MVD), Indian Driving and Cityscapes datasets, surpassing the previously best approach on MVD by +4.5% PQ and +5.2% mAP.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- EfficientPS: Efficient Panoptic SegmentationInternational Journal of Computer Vision, 2021
- Object-Contextual Representations for Semantic Segmentation2020
- SSAP: Single-Shot Instance Segmentation With Affinity PyramidIEEE Transactions on Circuits and Systems for Video Technology, 2020
- Deep High-Resolution Representation Learning for Visual RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
- Attention-Guided Unified Network for Panoptic Segmentation2019
- Panoptic Feature Pyramid Networks2019
- The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes2017
- RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation2017
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Theory of Maxima and the Method of LagrangeSIAM Journal on Applied Mathematics, 1971