Peer Is Your Pillar: A Data-Unbalanced Conditional GANs for Few-Shot Image Generation
- 23 October 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 35 (2) , 1303-1317
Abstract
Few-shot image generation aims to train generative models using a small number of training images. When there are few images available for training (e.g. 10 images), Learning From Scratch (LFS) methods often generate images that closely resemble the training data while Transfer Learning (TL) methods try to improve performance by leveraging prior knowledge from GANs pre-trained on large-scale datasets. However, current TL methods may not allow for sufficient control over the degree of knowledge preservation from the source model, making them unsuitable for setups where the source and target domains are not closely related. To address this, we propose a novel pipeline called Peer is your Pillar (PIP), which combines a target few-shot dataset with a peer dataset to create a data-unbalanced conditional generation. Our approach includes a class embedding method that separates the class space from the latent space, and we use a direction loss based on pre-trained CLIP to improve image diversity. Experiments on various few-shot datasets demonstrate the advancement of the proposed PIP, especially reduces the training requirements of few-shot image generation.Keywords
All Related Versions
Funding Information
- Jiangsu Basic Research Programs-Natural Science Foundation (BK20200039)
- Collaborative Innovation Center of Atmospheric Environment and Equipment Technology fund
- National Natural Science Foundation of China (61836011, 62172232, U19B2044, U22B2062)
This publication has 28 references indexed in Scilit:
- FakeCLR: Exploring Contrastive Learning for Solving Latent Discontinuity in Data-Efficient GANs2022
- Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment2022
- A Closer Look at Few-shot Image Generation2022
- Few-shot Image Generation via Cross-domain Correspondence2021
- Regularizing Generative Adversarial Networks under Limited Data2021
- Analyzing and Improving the Image Quality of StyleGAN2020
- MineGAN: Effective Knowledge Transfer From GANs to Target Domains With Few Images2020
- Style Transfer by Relaxed Optimal Transport and Self-Similarity2019
- A Style-Based Generator Architecture for Generative Adversarial Networks2019
- Transferring GANs: Generating Images from Limited Data2018