Phase Space Reconstruction Driven Spatio-Temporal Feature Learning for Dynamic Facial Expression Recognition

Authors:
Publisher website
Google Scholar
Abstract
Automatic Dynamic Facial Expression Recognition (DFER) is challenging since how to effectively capture facial temporal dynamics is still an open problem. As the variations of facial expressions is a dynamic system that satisfies underlying rules, it is essential to explore the fundamental temporal properties for recognizing dynamic expressions. Inspired by the phase space reconstruction method for time series analysis, we propose a Phase Space Reconstruction Network (PSRNet) for learning spatio-temporal facial features. First, 3D convolutional neural networks are used to extract spatial and short-term features, which indicate each frame's state termed as observations in the phase space. All the observations compose the trajectory of the dynamical system. Then, a data-driven across-correlation matrix is inferred to reveal the relationship of the observations. With this matrix, the phase space reconstruction module reconstructs the trajectory by aggregating the observations adaptively. Reconstructed observations represent the gradual process of dynamic facial expressions, which is beneficial to recognize these expressions. The experiment results on Oulu, MMI, and CK+ demonstrate that PSRNet can extract more informative and representative spatio-temporal features for DFER. Moreover, the visualization reveals that the reconstructed features have global consistency in facial regions and find the underlying evolutionary pattern of dynamic facial expression.
Funding Information
  • National Natural Science Foundation of China (61532009, 61825601)
  • Jiangsu Project (BK20192004)

This publication has 40 references indexed in Scilit: