Multimodal Self-Assessed Personality Estimation During Crowded Mingle Scenarios Using Wearables Devices and Cameras
- 23 July 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 13 (1) , 46-59
Abstract
This paper focuses on the automatic classification of self-assessed personality traits from the HEXACO inventory during crowded mingle scenarios. These scenarios provide rich study cases for social behavior analysis but are also challenging to analyze automatically as people in them interact dynamically and freely in an in-the-wild face-to-face setting. To do so, we leverage the use of wearable sensors recording acceleration and proximity, and video from overhead cameras. We use 3 different behavioral modality types (movement, speech and proximity) coming from 2 sensors (wearable and camera). Unlike other works, we extract an individual’s speaking status from a single body worn triaxial accelerometer instead of audio, which scales easily to large populations. Additionally, we study the effect of different combinations of modality types on the personality estimation, and how this relates to the nature of each trait. We also include an analysis of feature complementarity and an evaluation of feature importance for the classification, showing that combining complementary modality types further improves the classification performance. We estimate the self-assessed personality traits both using a binary classification (community’s standard) and as a regression over the trait scores. Finally, we analyze the impact of the accuracy of the speech detection on the overall performance of the personality estimation.Keywords
Funding Information
- Dutch national program COMMIT
- Instituto Tecnológico de Costa Rica
This publication has 39 references indexed in Scilit:
- The HEXACO Honesty-Humility, Agreeableness, and Emotionality FactorsPersonality and Social Psychology Review, 2014
- Social signal processing: Survey of an emerging domainImage and Vision Computing, 2009
- Automatic nonverbal analysis of social interaction in small groups: A reviewImage and Vision Computing, 2009
- Modeling the Personality of Participants During Group Interactions2009
- The prediction of Honesty–Humility-related criteria by the HEXACO and Five-Factor Models of personalityJournal of Research in Personality, 2008
- Recognising Human Emotions from Body Movement and Gesture Dynamics2007
- Personality in its natural habitat: Manifestations and implicit folk theories of personality in daily life.Journal of Personality and Social Psychology, 2006
- Psychometric Properties of the HEXACO Personality InventoryMultivariate Behavioral Research, 2004
- A Six-Factor Structure of Personality-Descriptive Adjectives: Solutions From Psycholexical Studies in Seven Languages.Journal of Personality and Social Psychology, 2004
- Two-Frame Motion Estimation Based on Polynomial Expansion2003