GAME: Learning Multimodal Interactions via Graph Structures for Personality Trait Estimation
Preprint
- 21 May 2025
- preprint
- Published by MDPI AG in Preprints.org
Abstract
Apparent personality analysis from short videos poses significant challenges due to the complex interplay of visual, auditory, and textual cues. In this paper, we propose GAME, a Graph-Augmented Multimodal Encoder designed to robustly model and fuse multi-source features for automatic personality prediction. For the visual stream, we construct a facial graph and introduce a dual-branch Geo Two-Stream Network, which combines Graph Convolutional Networks (GCNs) and Convolutional Neural Networks (CNNs) with attention mechanisms to capture both structural and appearance-based facial cues. Complementing this, global context and identity features are extracted using pretrained ResNet18 and VGGFace backbones. To capture temporal dynamics, frame-level features are processed by a BiGRU enhanced with temporal attention modules. Meanwhile, audio representations are derived from the VGGish network, and linguistic semantics are captured via the XLM-Roberta transformer. To achieve effective multimodal integration, we propose a Channel Attention-based Fusion module, followed by a Multi-Layer Perceptron (MLP) regression head for predicting personality traits. Extensive experiments show that GAME consistently outperforms existing methods across multiple benchmarks, validating its effectiveness and generalizability.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- MedConv: Convolutions Beat Transformers on Long-Tailed Bone Density Prediction2025
- CSCE: Boosting LLM Reasoning by Simultaneous Enhancing of Causal Significance and Consistency2025
- Efficient Knowledge Transfer in Multi-Task Learning through Task-Adaptive Low-Rank Representation2025
- Retraction Notice to "Graph-Driven Multimodal Feature Learning Framework for Apparent Personality Assessment"ICCK Transactions on Emerging Topics in Artificial Intelligence, 2025
- A multi-modal personality prediction systemKnowledge-Based Systems, 2022
- Modeling, Recognizing, and Explaining Apparent Personality From VideosIEEE Transactions on Affective Computing, 2020
- Deep Bimodal Regression of Apparent Personality Traits from Short Video SequencesIEEE Transactions on Affective Computing, 2017
- Multimodal First Impression Analysis with Deep Residual NetworksIEEE Transactions on Affective Computing, 2017
- Multimodal fusion of audio, scene, and face features for first impression estimation2016
- Deep Bimodal Regression for Apparent Personality Analysis2016