Localization Distillation for Dense Object Detection
Authors:
Top Cited Papers
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9397-9406
Abstract
Knowledge distillation (KD) has witnessed its powerful capability in learning compact models in object detection. Previous KD methods for object detection mostly focus on imitating deep features within the imitation regions instead of mimicking classification logit due to its inefficiency in distilling localization information and trivial improvement. In this paper, by reformulating the knowledge distillation process on localization, we present a novel localization distillation (LD) method which can efficiently transfer the localization knowledge from the teacher to the student. Moreover, we also heuristically introduce the concept of valuable localization region that can aid to selectively distill the semantic and localization knowledge for a certain region. Combining these two new components, for the first time, we show that logit mimicking can outperform feature imitation and localization knowledge distillation is more important and efficient than semantic knowledge for distilling object detectors. Our distillation scheme is simple as well as effective and can be easily applied to different dense object detectors. Experiments show that our LD can boost the AP score of GFocal-ResNet-50 with a single-scale 1 x training schedule from 40.1 to 42.1 on the COCO benchmark without any sacrifice on the inference speed. Our source code and pretrained models are publicly available at https://github.com/HikariTJU/LD.Keywords
Funding Information
- National Key Research and Development Program of China (2018AAA0100400)
- NSFC (61922046,62172127)
This publication has 45 references indexed in Scilit:
- Feature Pyramid Networks for Object Detection2017
- YOLO9000: Better, Faster, Stronger2017
- Mimicking Very Efficient Network for Object Detection2017
- Aggregated Residual Transformations for Deep Neural Networks2017
- UnitBox2016
- Deep Residual Learning for Image Recognition2016
- LocNet: Improving Localization Accuracy for Object Detection2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- Object Detection with Discriminatively Trained Part-Based ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009