AMD: Adaptive Masked Distillation for Object Detection
Authors:
- 18 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Abstract
As a general model compression paradigm, feature-based knowledge distillation allows the student model to learn expressive features from the teacher counterpart. In this paper, we mainly focus on designing an effective feature-distillation framework and propose a spatial-channel adaptive masked distillation (AMD) network for object detection. More specifically, in order to accurately reconstruct important feature regions, we first perform attention-guided feature masking on the feature map of the student network, such that we can identify the important features via spatially adaptive feature masking instead of random masking in the previous methods. In addition, we employ a simple and efficient module to allow the student network channel to be adaptive, improving its model capability in object perception and detection. In contrast to the previous methods, more crucial object-aware features can be reconstructed and learned from the proposed network, which is conducive to accurate object detection. The empirical experiments demonstrate the superiority of our method: with the help of our proposed distillation method, the student networks report 41.3%, 42.4%, and 42.7% mAP scores when RetinaNet, Cascade Mask-RCNN and RepPoints are respectively used as the teacher framework for object detection, which outperforms the previous state-of-the-art distillation methods including FGD and MGD.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Cascade R-CNN: High Quality Object Detection and Instance SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2019
- RepPoints: Point Set Representation for Object Detection2019
- Distilling Object Detectors With Fine-Grained Feature Imitation2019
- Squeeze-and-Excitation Networks2018
- Focal Loss for Dense Object Detection2017
- Mask R-CNN2017
- Mimicking Very Efficient Network for Object Detection2017
- HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection2016
- Fast R-CNN2015
- Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model2015