LES-Talker: Fine-Grained Emotion Editing for Talking Head Generation in Linear Emotion Space
- 23 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 17 (2) , 1695-1708
Abstract
While existing one-shot talking head generation models have achieved progress in coarse-grained emotion editing, there is still a lack of fine-grained emotion editing models with high interpretability. We argue that for an approach to be considered fine-grained, it needs to provide clear definitions and sufficiently detailed differentiation. We present LES-Talker, a novel one-shot talking head generation model with high interpretability, to achieve fine-grained emotion editing across emotion types, emotion levels, and facial units. We propose a Linear Emotion Space (LES) definition based on Facial Action Units to characterize emotion transformations as vector transformations. We design the Cross-Dimension Attention Net (CDAN) to deeply mine the correlation between LES representation and 3D model representation. Through mining multiple relationships across different feature and structure dimensions, we enable LES representation to guide the controllable deformation of 3D model. In order to adapt the multimodal data with deviations to the LES and enhance visual quality, we utilize specialized network design and training strategies. Experiments show that our method provides high visual quality along with multilevel and inter pretable fine-grained emotion editing, outperforming mainstream methods. Project page: https://peterfanfan.github.io/LES-Talker/Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62272364)
- National Natural Science Foundation of China (62472342)
- Provincial Key Research and Development Program of Shaanxi (2024GH-ZDXM-47)
- Higher Education Science Research Planning Project of China Association of Higher Education (24PG0101)
- China Adult Education Association Project (2023-886ZB)
- Shaanxi Higher Education Teaching Reform Research Project (25BY055)
- China Education Society of Electronics Teaching Reform Project (DJZ25005)
- Teaching Reform Project of the Chinese Society for Educational Development Strategy (ZLB20250728)
- Xidian University Graduate Teaching Reform Project (AIZS2501)
This publication has 56 references indexed in Scilit:
- TalkingGaussian: Structure-Persistent 3D Talking Head Synthesis via Gaussian Splatting2024
- EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesis2024
- Facial Expression-Aware Talking Head Generation with 3D Morphable Model2024
- Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation2021
- One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing2021
- Audio-Driven Emotional Video Portraits2021
- Flow-guided One-shot Talking Face Generation with a High-resolution Audio-visual Dataset2021
- MEAD: A Large-Scale Audio-Visual Dataset for Emotional Talking-Face Generation2020
- Out of Time: Automated Lip Sync in the Wild2017
- Facial Action Coding System1978