Zhongyi Fan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Zhongyi Fan has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- Gang Li
University of the Chinese Academy of Sciences & Institute of Software, Chinese Academy of Sciences, Beijing, Beijing, China
1 shared publication - Heliang Zheng
University of Science and Technology of China, Hefei, Anhui, China
1 shared publication - Yibing Zhan
Wuhan University
1 shared publication - Zixin Yin
Beihang University, Beijing, China
1 shared publication
Latest Publications
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 11013-11021
DreamBooth++: Boosting Subject-Driven Generation via Region-Level References Packing
DreamBooth has demonstrated significant potential in subject-driven text-to-image generation, especially in scenarios requiring precise preservation of a subject's appearance. However, it still suffers from inefficiency and requires extensive iterative training to customize concepts using a small set of reference images. To address these issues, we introduce DreamBooth++, a region-level training strategy designed to significantly improve the efficiency and effectiveness of learning specific subjects. In particular, our approach employs a region-level data re-formulation technique that packs a set of reference images into a single sample, significantly reducing computational costs. Moreover, we adapt convolution and self-attention layers to ensure their processings are restricted within individual regions. Thus their operational scope (i.e., receptive field) can be preserved within a single subject, avoiding generating multiple sub-images within a single image. Last but not least, we design a text-guided prior regularization between our model and the pretrained one to preserve the original semantic generation ability. Comprehensive experiments demonstrate that our training strategy not only accelerates the subject-learning process but also significantly boosts fidelity to both subject and prompts in subject-driven generation.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 11013-11021
Open AccessDreamBooth++: Boosting Subject-Driven Generation via Region-Level References Packing
DreamBooth has demonstrated significant potential in subject-driven text-to-image generation, especially in scenarios requiring precise preservation of a subject's appearance. However, it still suffers from inefficiency and requires extensive iterative training to customize concepts using a small set of reference images. To address these issues, we introduce DreamBooth++, a region-level training strategy designed to significantly improve the efficiency and effectiveness of learning specific subjects. In particular, our approach employs a region-level data re-formulation technique that packs a set of reference images into a single sample, significantly reducing computational costs. Moreover, we adapt convolution and self-attention layers to ensure their processings are restricted within individual regions. Thus their operational scope (i.e., receptive field) can be preserved within a single subject, avoiding generating multiple sub-images within a single image. Last but not least, we design a text-guided prior regularization between our model and the pretrained one to preserve the original semantic generation ability. Comprehensive experiments demonstrate that our training strategy not only accelerates the subject-learning process but also significantly boosts fidelity to both subject and prompts in subject-driven generation.