Information Symmetry Matters: A Modal-Alternating Propagation Network for Few-Shot Learning
- 20 January 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 31 (10577149) , 1520-1531
Abstract
Semantic information provides intra-class consistency and inter-class discriminability beyond visual concepts, which has been employed in Few-Shot Learning (FSL) to achieve further gains. However, semantic information is only available for labeled samples but absent for unlabeled samples, in which the embeddings are rectified unilaterally by guiding the few labeled samples with semantics. Therefore, it is inevitable to bring a cross-modal bias between semantic-guided samples and nonsemantic-guided samples, which results in an information asymmetry problem. To address this problem, we propose a Modal-Alternating Propagation Network (MAP-Net) to supplement the absent semantic information of unlabeled samples, which builds information symmetry among all samples in both visual and semantic modalities. Specifically, the MAP-Net transfers the neighbor information by the graph propagation to generate the pseudo-semantics for unlabeled samples guided by the completed visual relationships and rectify the feature embeddings. In addition, due to the large discrepancy between visual and semantic modalities, we design a Relation Guidance (RG) strategy to guide the visual relation vectors via semantics so that the propagated information is more beneficial. Extensive experimental results on three semantic-labeled datasets, i.e., Caltech-UCSD-Birds 200-2011, SUN Attribute Database and Oxford 102 Flower, have demonstrated that our proposed method achieves promising performance and outperforms the state-of-the-art approaches, which indicates the necessity of information symmetry.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62176178, 61632018)
This publication has 39 references indexed in Scilit:
- Heterogeneous Graph Neural Network2019
- Multi-Level Semantic Feature Augmentation for One-Shot LearningIEEE Transactions on Image Processing, 2019
- Learning to Compare: Relation Network for Few-Shot Learning2018
- Low-Shot Learning from Imaginary Data2018
- Spatial–Temporal Recurrent Neural Network for Emotion RecognitionIEEE Transactions on Cybernetics, 2018
- Emotion Recognition in Context2017
- The SUN Attribute Database: Beyond Categories for Deeper Scene UnderstandingInternational Journal of Computer Vision, 2014
- Automated Flower Classification over a Large Number of Classes2008
- Learning to recognize objectsTrends in Cognitive Sciences, 1999
- On beyond Zebra: The relation of linguistic and visual informationCognition, 1987