StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generation
Top Cited Papers
- 1 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Signal Processing
- Vol. 20 (2) , 192-201
Abstract
The recent advancements in image-text diffusion models have stimulated research interest in large-scale 3D generative models. Nevertheless, the limited availability of diverse 3D resources presents significant challenges to learning. In this paper, we present a novel method for generating high-quality, stylized 3D avatars that utilizes pre-trained image-text diffusion models for data generation and a Generative Adversarial Network (GAN)-based 3D generation network for training. Our method leverages the comprehensive priors of appearance and geometry offered by image-text diffusion models to generate multi-view images of avatars in various styles. During data generation, we employ poses extracted from existing 3D models to guide the generation of multi-view images. To handle inaccurate pose annotations of stylized images, we investigate view-specific prompts and develop a coarse-to-fine discriminator for GAN training. Additionally, we develop a latent diffusion model within the style space of StyleGAN to enable the generation of avatars based on image or text inputs. Our approach demonstrates superior performance over current state-of-the-art methods in terms of visual quality and diversity of the produced avatars.Keywords
Funding Information
- National Natural Science Foundation of China (62506307)
- Zhejiang Leading Innovative and Entrepreneur Team Introduction Program (2024R01007)
This publication has 42 references indexed in Scilit:
- LGM: Large Multi-view Gaussian Model for High-Resolution 3D Content Creation2024
- Diffusion Autoencoders: Toward a Meaningful and Decodable Representation2022
- Zero-Shot Text-Guided Object Generation with Dream Fields2022
- Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields2022
- GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields2021
- Diffusion Probabilistic Models for 3D Point Cloud Generation2021
- pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis2021
- Analyzing and Improving the Image Quality of StyleGAN2020
- A Style-Based Generator Architecture for Generative Adversarial Networks2019
- Facial Landmark Detection by Deep Multi-task Learning2014