SonicFace
Authors:
Open Access
- 27 December 2021
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 5 (4) , 1-33
Abstract
Accurate recognition of facial expressions and emotional gestures is promising to understand the audience's feedback and engagement on the entertainment content. Existing methods are primarily based on various cameras or wearable sensors, which either raise privacy concerns or demand extra devices. To this aim, we propose a novel ubiquitous sensing system based on the commodity microphone array --- SonicFace, which provides an accessible, unobtrusive, contact-free, and privacy-preserving solution to monitor the user's emotional expressions continuously without playing hearable sound. SonicFace utilizes a pair of speaker and microphone array to recognize various fine-grained facial expressions and emotional hand gestures by emitted ultrasound and received echoes. Based on a set of experimental evaluations, the accuracy of recognizing 6 common facial expressions and 4 emotional gestures can reach around 80%. Besides, the extensive system evaluations with distinct configurations and an extended real-life case study have demonstrated the robustness and generalizability of the proposed SonicFace system.Keywords
This publication has 66 references indexed in Scilit:
- Predicting Ad Liking and Purchase Intent: Large-Scale Analysis of Facial Responses to AdsIEEE Transactions on Affective Computing, 2014
- Ratings for emotion film clipsBehavior Research Methods, 2014
- Emotion-Induced Engagement in Internet Video AdvertisementsJournal of Marketing Research, 2012
- Looking at the viewer: analysing facial activity to detect personal highlights of multimedia contentsMultimedia Tools and Applications, 2010
- Efficient voice activity detection algorithms using long-term speech informationSpeech Communication, 2004
- Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant.Journal of Personality and Social Psychology, 1999
- Innate and universal facial expressions: Evidence from developmental and cross-cultural research.Psychological Bulletin, 1994
- Spatio-temporal spectral analysis by eigenstructure methodsIEEE Transactions on Acoustics, Speech, and Signal Processing, 1984
- A circumplex model of affect.Journal of Personality and Social Psychology, 1980
- What Emotion Categories Can Observers Judge from Facial Behavior?1972