TADA! Text to Animatable Digital Avatars
Authors:
Top Cited Papers
- 18 March 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1508-1519
Abstract
We introduce TADA, a simple-yet-effective approach that takes textual descriptions and produces expressive 3D avatars with high-quality geometry and lifelike textures, that can be animated and rendered with traditional graphics pipelines. Existing text-based character generation methods are limited in terms of geometry and texture quality, and cannot be realistically animated due to the misalignment between the geometry and the texture, particularly in the face region. To address these limitations, TADA leverages the synergy of a 2D diffusion model and a parametric body model. Specifically, we derive a high-resolution upsampled version of SMPL-X with a displacement layer and a texture map, and use hierarchical rendering with score distillation sampling (SDS) to create high-quality, detailed, holistic 3D avatars from text. To ensure alignment between the geometry and texture, we render normals and RGB images of the generated character and exploit their latent embeddings during the SDS optimization process. We further drive the character’s face with multiple expressions during optimization, ensuring that its semantics remain consistent with the original SMPL-X model. Both qualitative and quantitative evaluations show that TADA significantly surpasses existing approaches. TADA enables large-scale creation of digital characters ready for animation and rendering, while also enabling text-guided editing. The code is public for research purposes at tada.is.tue.mpg.deKeywords
All Related Versions
Funding Information
- Ministry of Education
This publication has 55 references indexed in Scilit:
- TeCH: Text-Guided Reconstruction of Lifelike Clothed Humans2024
- DiffuStereo: High Quality Human Reconstruction via Diffusion-Based Stereo Using Sparse Cameras2022
- StyleGAN-Human: A Data-Centric Odyssey of Human Generation2022
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- Monocular Real-Time Volumetric Performance Capture2020
- Reconstructing NBA Players2020
- DeepCap: Monocular Human Performance Capture Using Weak Supervision2020
- PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization2020
- DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation2019
- Expressive Body Capture: 3D Hands, Face, and Body From a Single Image2019