Abstract
The growing demand for multimodal functionalities in consumer electronics has driven the widespread adoption of multimodal pre-trained models, such as CLIP, enhancing human-computer interactions on edge devices through adaptive and context-aware functionalities. However, deep neural architectures impose significant computational complexity, storage demands, and data privacy challenges that limit traditional finetuning methods at the edge. To overcome these constraints, we propose an efficient knowledge transfer framework comprising two main modules: a data filtering module employing a combined confidence-driven and random sampling strategy to select representative samples while ensuring privacy compliance, and a knowledge distillation module that explicitly transfers multimodal representations via contrastive learning. Specifically, we construct bidirectional cross-modal contrastive distributions and minimize their Kullback-Leibler divergence to achieve representation alignment between the old and new models. Experimental results indicate that our method significantly outperforms conventional fine-tuning, achieving more than 1.5 accuracy improvement using only 5% of the original training data. Additionally, we identify optimal data selection strategies tailored to different data availability scenarios, demonstrating the practical effectiveness and flexibility of our approach for resource-constrained edge computing applications.
Funding Information
  • National Natural Science Foundation of China (62206015, 62227801)
  • Fundamental Research Funds for the Central Universities (FRF-KST-25-008)
  • Young Scientist Program of The National New Energy Vehicle Technology Innovation Center