ConsistentID: Portrait Generation With Multimodal Fine-Grained Identity Preserving
Top Cited Papers
- 12 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (5) , 5639-5654
Abstract
Diffusion-based technologies have made significant strides, particularly in personalized and customized facial generation. However, existing methods struggle to achieve high-fidelity and detailed identity (ID) consistency. This is mainly due to two challenges: insufficient fine-grained control over specific facial areas and the absence of a comprehensive strategy for ID preservation that accounts for both intricate facial details and the overall facial structure. To address these limitations, we introduce ConsistentID, an innovative method crafted for diverse identity-preserving portrait generation under fine-grained multimodal facial prompts, utilizing only a single reference image. ConsistentID comprises two core components: a multimodal facial prompt generator and an ID-preservation network. The facial prompt generator combines localized facial features, facial feature descriptions, and overall facial descriptions to enhance the precision of facial detail reconstruction. The ID-preservation network, optimized with a facial attention localization strategy, ensures consistent identity preservation across facial regions. Together, these components leverage fine-grained multimodal identity information to improve identity preservation accuracy significantly. To drive ConsistentID’s training, we propose a fine-grained portrait dataset, FGID, with over 500,000 facial images, offering greater diversity and comprehensiveness than existing public facial datasets. Experimental results substantiate that our ConsistentID achieves exceptional precision and diversity in personalized facial generation, surpassing existing methods in the MyStyle dataset. In addition, although ConsistentID introduces more multimodal ID information, it still maintains rapid inference speed during the generation process.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62372482)
- National Natural Science Foundation Youth Basic Research Program (623B2100)
- National Natural Science Foundation of China (62476293)
- National Key Research and Development Program of China (2024YFE0203100)
- UAEU Start-Up Research (G00005318)
- UAEU-ZU Joint Research Program (G00005305)
This publication has 43 references indexed in Scilit:
- DiffFace: Diffusion-based face swapping with facial guidancePattern Recognition, 2025
- Arc2Face: A Foundation Model for ID-Consistent Human Faces2024
- FastComposer: Tuning-Free Multi-subject Image Generation with Localized AttentionInternational Journal of Computer Vision, 2024
- Mask-FPAN: Semi-supervised face parsing in the wild with de-occlusion and UV GANComputers & Graphics, 2023
- Detailed feature extraction network-based fine-grained face segmentationKnowledge-Based Systems, 2022
- MEAD: A Large-Scale Audio-Visual Dataset for Emotional Talking-Face Generation2020
- Text2FaceGAN: Face Generation from Fine Grained Textual Descriptions2019
- BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation2018
- Toward Characteristic-Preserving Image-Based Virtual Try-On Network2018
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015