Domain-Invariant Progressive Knowledge Distillation for UAV-Based Object Detection
- 6 November 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 22 (1545598X) , 1-5
Abstract
Knowledge distillation (KD) is an effective method for compressing models in object detection tasks. Due to limited computational capability, unmanned aerial vehicle-based object detection (UAV-OD) widely adopt the KD technique to obtain lightweight detectors. Existing methods often overlook the significant differences in feature space caused by the large gap in scale between the teacher and student models. This limitation hampers the efficiency of knowledge transfer during the distillation process. Furthermore, the complex backgrounds in aerial images make it challenging for the student model to efficiently learn the object features. In this letter, we propose a novel KD framework for UAV-OD. Specifically, a progressive distillation approach is designed to alleviate the feature gap between teacher and student models. Then, a new feature alignment method is provided to extract object-related features for enhancing the student model’s knowledge reception efficiency. Finally, extensive experiments are conducted to validate the effectiveness of our proposed approach. The results demonstrate that our proposed method achieves state-of-the-art performance on two datasets.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62372155)
- National Natural Science Foundation of China (62302149)
- Postgraduate Research and Practice Innovation Program of Jiangsu Province (SJCX24_0183)
- Fundamental Research Funds for the Central Universities (B240201077)
- Chinese Aeronautical Establishment (2022Z071108001)
- Joint Fund of Ministry of Education for Equipment Pre-Researchunder (8091B022123)
- Water Science and Technology Project of Jiangsu Province (2021063)
- QingLan Project of Jiangsu Province
- Changzhou Science and Technology under Project
This publication has 28 references indexed in Scilit:
- Direct Distillation Between Different Domains2024
- TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models2023
- Generalized UAV Object Detection via Frequency Domain Disentanglement2023
- Focal and Global Knowledge Distillation for Detectors2022
- QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection2022
- Single-Domain Generalized Object Detection in Urban Scene via Cyclic-Disentangled Self-Distillation2022
- A Fourier-based Framework for Domain Generalization2021
- Harmonizing Transferability and Discriminability for Adapting Object Detectors2020
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- The Fast Fourier Transform1982