MagicNaming: Consistent Identity Generation by Finding a “Name Space” in T2I Diffusion Models
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (10) , 10439-10447
Abstract
Large-scale text-to-image diffusion models, (e.g., DALL-E, SDXL) are capable of generating famous persons by simply referring to their names. Is it possible to make such models generate generic identities as simple as the famous ones, e.g., just use a name? In this paper, we explore the existence of a ``Name Space'', where any point in the space corresponds to a specific identity. Fortunately, we find some clues in the feature space spanned by text embedding of celebrities' names. Specifically, we first extract the embeddings of celebrities' names in the Laion5B dataset with the text encoder of diffusion models. Such embeddings are used as supervision to learn an encoder that can predict the name (actually an embedding) of a given face image. We experimentally find that such name embeddings work well in promising the generated image with good identity consistency. Note that like the names of celebrities, our predicted name embeddings are disentangled from the semantics of text inputs, making the original generation capability of text-to-image models well-preserved. Moreover, by simply plugging such name embeddings, all variants (e.g., from Civitai) derived from the same base model (i.e., SDXL) readily become identity-aware text-to-image models.Keywords
This publication has 16 references indexed in Scilit:
- FastComposer: Tuning-Free Multi-subject Image Generation with Localized AttentionInternational Journal of Computer Vision, 2024
- PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding2024
- PortraitBooth: A Versatile Portrait Model for Fast Identity-Preserved Personalization2024
- BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion2023
- DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation2023
- Multi-Concept Customization of Text-to-Image Diffusion2023
- Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation2023
- MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation2023
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- Blended Diffusion for Text-driven Editing of Natural Images2022