A Selective Expression Manipulation With Parametric 3D Facial Model
Open Access
- 8 February 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 17066-17084
Abstract
This paper proposes a novel method to represent expressive 3D facial shapes called the Selective Expression Manipulation (SEM) by fitting the expression coefficients of delta-blendshapes, which is the standard parametric facial model widely used in industries. SEM focuses on preserving blendshape semantics to characterize the facial shapes since the facial shape obtained by minimizing the distance to sparse facial landmarks might fail to signify a facial expression from a human being’s perspective. Assuming each delta-blendshape corresponds to a facial movement with semantic meaning, SEM finds a series of facial motions required to compose the target facial expression. In addition, SEM sequentially determines a sufficient number of expressions and coefficients closely resembling the target facial movements by introducing similarities to quantify the directional correlation of facial motions between a target and a blendshape, excluding redundant expressions in terms of motions from the neutral shape. As a result, far fewer inter-correlated expressions that significantly increase the target correlation can be obtained. Furthermore, SEM exhibits substantial improvement in accuracy, correlation, semantics, and stability in experiments over previous facial fitting schemes and state-of-the-art methods. It is demonstrated that SEM enables accurate and realistic 3D facial shape generation by semantically manipulating expression delta-blendshapes.Keywords
Funding Information
- National Research Foundation of Korea (NRF) grant
- Korea government (Ministry of Science and ICT (2020R1A2C3011697, 2022R1F1A1068704)
- Yonsei University Research Fund of 2021 (2021-22-0001)
- Sookmyung Women’s University Research Grants (No. 1-2203-2043)
This publication has 80 references indexed in Scilit:
- A Practical Approach to Direct Manipulation BlendshapesJournal of Graphics Tools, 2012
- Spacetime expression cloning for blendshapesACM Transactions on Graphics, 2012
- Performance Analysis of $l_0$ Norm Constraint Least Mean Square AlgorithmIEEE Transactions on Signal Processing, 2012
- Realtime performance-based facial animationACM Transactions on Graphics, 2011
- High-quality passive facial performance capture using anchor framesACM Transactions on Graphics, 2011
- High resolution passive facial performance captureACM Transactions on Graphics, 2010
- Example-based facial riggingACM Transactions on Graphics, 2010
- Algorithms for simultaneous sparse approximation. Part I: Greedy pursuitSignal Processing, 2006
- An interior algorithm for nonlinear optimization that combines line search and trust region stepsMathematical Programming, 2005
- The Mahalanobis distanceChemometrics and Intelligent Laboratory Systems, 2000