Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 11194-11203
Abstract
Training a generative adversarial network (GAN) with limited data has been a challenging task. A feasible solution is to start with a GAN well-trained on a large scale source domain and adapt it to the target domain with a few samples, termed as few shot generative model adaption. However, existing methods are prone to model overfitting and collapse in extremely few shot setting (less than 10). To solve this problem, we propose a relaxed spatial structural alignment (RSSA) method to calibrate the target generative models during the adaption. We design a cross-domain spatial structural consistency loss comprising the self-correlation and disturbance correlation consistency loss. It helps align the spatial structural information between the synthesis image pairs of the source and target domains. To relax the cross-domain alignment, we compress the original latent space of generative models to a subspace. Image pairs generated from the subspace are pulled closer. Qualitative and quantitative experiments show that our method consistently surpasses the state-of-the-art methods in few shot setting. Our source code: https://github.com/StevenShaw1999/RSSA.Keywords
All Related Versions
Funding Information
- National Key R&D Program of China (2018AAA0102000)
- National Natural Science Foundation of China (U21B2038,61931008,61732007)
This publication has 26 references indexed in Scilit:
- Transferring GANs: Generating Images from Limited Data2018
- Generative Image Inpainting with Contextual Attention2018
- Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
- Semantic Image Inpainting with Deep Generative Models2017
- Image-to-Image Translation with Conditional Adversarial Networks2017
- Distributed image understanding with semantic dictionary and semantic expansionNeurocomputing, 2016
- Fast R-CNN2015
- Holistically-Nested Edge Detection2015
- Face Photo-Sketch Synthesis and RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2008
- Measures of the Amount of Ecologic Association Between SpeciesEcology, 1945