A Closer Look at Few-shot Image Generation
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9130-9140
Abstract
Modern GANs excel at generating high quality and diverse images. However, when transferring the pretrained GANs on small target data (e.g., 10-shot), the generator tends to replicate the training samples. Several methods have been proposed to address this few-shot image generation task, but there is a lack of effort to analyze them under a unified framework. As our first contribution, we propose a framework to analyze existing methods during the adaptation. Our analysis discovers that while some methods have disproportionate focus on diversity preserving which impede quality improvement, all methods achieve similar quality after convergence. Therefore, the better methods are those that can slow down diversity degradation. Furthermore, our analysis reveals that there is still plenty of room to further slow down diversity degradation. Informed by our analysis and to slow down the diversity degradation of the target generator during adaptation, our second contribution proposes to apply mutual information (MI) maximization to retain the source domain's rich multi-level diversity information in the target domain generator. We propose to perform MI maximization by contrastive loss (CL), leverage the generator and discriminator as two feature encoders to extract different multi-level features for computing CL. We refer to our method as Dual Contrastive Learning (DCL). Extensive experiments on several public datasets show that, while leading to a slower diversity-degrading generator during adaptation, our proposed DCL brings visually pleasant quality and state-of-the-art quantitative performance.Keywords
All Related Versions
Funding Information
- National Research Foundation, Singapore (AISG2-RP-2021-021)
This publication has 31 references indexed in Scilit:
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- Unsupervised Feature Learning via Non-parametric Instance Discrimination2018
- Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences, 2017
- 3D Object Representations for Fine-Grained Categorization2013
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009
- ImageNet: A large-scale hierarchical image database2009
- Face Photo-Sketch Synthesis and RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2008
- One-shot learning of object categoriesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2006
- Estimation of Entropy and Mutual InformationNeural Computation, 2003