Scale-Equivalent Distillation for Semi-Supervised Object Detection
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 14502-14511
Abstract
Recent Semi-Supervised Object Detection (SS-OD) methods are mainly based on self-training, i.e., generating hard pseudo-labels by a teacher model on unlabeled data as supervisory signals. Although they achieved certain success, the limited labeled data in semi-supervised learning scales up the challenges of object detection. We analyze the challenges these methods meet with the empirical experiment results. We find that the massive False Negative samples and inferior localization precision lack consideration. Besides, the large variance of object sizes and class imbalance (i.e., the extreme ratio between back-ground and object) hinder the performance of prior arts. Further, we overcome these challenges by introducing a novel approach, Scale-Equivalent Distillation (SED), which is a simple yet effective end-to-end knowledge distillation framework robust to large object size variance and class imbalance. SED has several appealing benefits compared to the previous works. (1) SED imposes a consistency regularization to handle the large scale variance problem. (2) SED alleviates the noise problem from the False Negative samples and inferior localization precision. (3) A re-weighting strategy can implicitly screen the potential foreground regions of the unlabeled data to reduce the effect of class imbalance. Extensive experiments show that SED consistently outperforms the recent state-of-the-art methods on different datasets with significant margins. For example, it surpasses the supervised counterpart by more than 10 mAP when using 5% and 10% labeled data on MS-COCO.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Gradient Harmonized Single-Stage DetectorProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised LearningIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Data Distillation: Towards Omni-Supervised Learning2018
- Deep Mutual Learning2018
- Focal Loss for Dense Object Detection2017
- Mask R-CNN2017
- Revisiting Unreasonable Effectiveness of Data in Deep Learning Era2017
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- ImageNet: A large-scale hierarchical image database2009
- Acceleration of Stochastic Approximation by AveragingSIAM Journal on Control and Optimization, 1992