Category Name Expansion and an Enhanced Multimodal Fusion Framework for Few-Shot Learning

Authors:
Open Access
Publisher website
Google Scholar
Abstract
With the advancement of image processing techniques, few-shot learning (FSL) has gradually become a key approach to addressing the problem of data scarcity. However, existing FSL methods often rely on unimodal information under limited sample conditions, making it difficult to capture fine-grained differences between categories. To address this issue, we propose a multimodal few-shot learning method based on category name expansion and image feature enhancement. By integrating the expanded category text with image features, the proposed method enriches the semantic representation of categories and enhances the model’s sensitivity to detailed features. To further improve the quality of cross-modal information transfer, we introduce a cross-modal residual connection strategy that aligns features across layers through progressive fusion. This approach enables the fused representations to maximize mutual information while reducing redundancy, effectively alleviating the information bottleneck caused by uneven entropy distribution between modalities and enhancing the model’s generalization ability. Experimental results demonstrate that our method achieves superior performance on both natural image datasets (CIFAR-FS and FC100) and a medical image dataset.
Funding Information
  • Key R&D Program of Shandong Province, China (2024CXGC010101, 202333044)
  • 20 Guidelines for New Colleges in Jinan City (2024CXGC010101, 202333044)

This publication has 21 references indexed in Scilit: