Preprint
Abstract
Letting a deep network be aware of the quality of its own predictions is an interesting yet important problem. In the task of instance segmentation, the confidence of instance classification is used as mask quality score in most instance segmentation frameworks. However, the mask quality, quantified as the IoU between the instance mask and its ground truth, is usually not well correlated with classification score. In this paper, we study this problem and propose Mask Scoring R-CNN which contains a network block to learn the quality of the predicted instance masks. The proposed network block takes the instance feature and the corresponding predicted mask together to regress the mask IoU. The mask scoring strategy calibrates the misalignment between mask quality and mask score, and improves instance segmentation performance by prioritizing more accurate mask predictions during COCO AP evaluation. By extensive evaluations on the COCO dataset, Mask Scoring R-CNN brings consistent and noticeable gain with different models, and outperforms the state-of-the-art Mask R-CNN. We hope our simple and effective approach will provide a new direction for improving instance segmentation. The source code of our method is available at \url{https://github.com/zjhuang22/maskscoring_rcnn}.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- CCNet: Criss-Cross Attention for Semantic SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
- Object Detection in Videos by High Quality Object LinkingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2019
- MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features2018
- Improving Object Localization with Fitness NMS and Bounded IoU Loss2018
- Proposal-Free Network for Instance-Level Object SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Deep patch learning for weakly supervised object classification and discoveryPattern Recognition, 2017