Semi-Identical Twins Variational AutoEncoder for Few-Shot Learning
- 9 January 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (7) , 9455-9469
Abstract
Data augmentation is a popular way for few-shot learning (FSL). It generates more samples as supplements and then transforms the FSL task into a common supervised learning problem for a solution. However, most data-augmentation-based FSL approaches only consider the prior visual knowledge for feature generation, thereby leading to low diversity and poor quality of generated data. In this study, we attempt to address this issue by incorporating both prior visual and prior semantic knowledge to condition the feature generation process. Inspired by some genetic characteristics of semi-identical twins, a novel multimodal generative FSL approach was developed named semi-identical twins variational autoencoder (STVAE) to better exploit the complementarity of these modality information by considering the multimodal conditional feature generation process as a process that semi-identical twins are born and collaborate to simulate their father. STVAE conducts feature synthesis by pairing two conditional variational autoencoders (CVAEs) with the same seed but different modality conditions. Subsequently, the generated features of two CVAEs are considered as semi-identical twins and adaptively combined to yield the final feature, which is considered as their fake father. STVAE requires that the final feature can be converted back into its paired conditions while ensuring these conditions remain consistent with the original in both representation and function. Moreover, STVAE is able to work in the partial modality-absence case due to the adaptive linear feature combination strategy. STVAE essentially provides a novel idea to exploit the complementarity of different modality prior information inspired by genetics in FSL. Extensive experimental results demonstrate that our work achieves promising performances in comparison to the recent state-of-the-art approaches, as well as validate its effectiveness on FSL under various modality settings.Keywords
Funding Information
- National Natural Science Foundation of China (62176030)
- Natural Science Foundation of Chongqing Municipality (cstc2021jcyj-msxmX0568)
This publication has 41 references indexed in Scilit:
- Few-Shot Learning With Geometric ConstraintsIEEE Transactions on Neural Networks and Learning Systems, 2020
- Multi-Level Semantic Feature Augmentation for One-Shot LearningIEEE Transactions on Image Processing, 2019
- Molecular Support for Heterogonesis Resulting in Sesquizygotic TwinningNew England Journal of Medicine, 2019
- Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the UglyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Learning to Compare: Relation Network for Few-Shot Learning2018
- Low-Shot Learning with Imprinted Weights2018
- Low-Shot Learning from Imaginary Data2018
- Generalized Zero-Shot Learning via Synthesized Examples2018
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Capturing Long-Tail Distributions of Object Subcategories2014