Xingzhi Wang

Group Emotion Recognition (GER) plays a vital role in public safety by enabling proactive risk warnings and resource allocation. However, current approaches often rely solely on visual emotional cues, and their performance is limited by linear fusion mechanisms. To overcome these limitations, this paper introduces a novel multimodal fusion framework that leverages Kolmogorov‐Arnold Networks (KANs) to effectively integrate multisource information. Our approach begins by extracting emotional features from three distinct modalities: group image scenes are processed by ResNet‐50, facial regions are analysed using VGGFace and textual descriptions, generated by the Gemini 2.0 model, are encoded with ELECTRA‐base. These features are then fused through an innovative hybrid‐level strategy based on KAN, which leverages its superior nonlinear modelling capability to generate robust emotional representations. Extensive experiments on three benchmark datasets, GAF 2.0, GAF 3.0 and GroupEmoW, demonstrate the consistent superiority of our method, which achieves state‐of‐the‐art accuracies of 92.87%, 92.01% and 91.88%, respectively. The advantage of the KAN‐based fusion is further validated through a comparison against seven alternative fusion mechanisms, paired statistical significance tests and an interpretability analysis of the learned modality contributions; moreover, we show that an open‐source MLLM (Qwen3‐VL) can replace the proprietary model with only a modest accuracy trade‐off, improving reproducibility and deployability.

Research on social psychology has revealed the existence of an affective mechanism in a human group, which is the group members spread their emotions to one another, the emotions of the group members form the group emotion, and the group emotion as a powerful force shapes the group members' emotions. Current group emotion recognition methods focus on how the emotions of the group members form the group-level emotion but rarely take into account how the group emotion feeds back to the group members instantaneously. This paper proposes a new graph convolutional network architecture to characterize this unique affective mechanism for group emotion recognition. We regard the group members as the nodes of the graph and introduce a pseudo node into the graph to represent the role of the group. This paper uses graph convolutional networks to model the emotional interactions within the group from a static image and constructs an effective emotional representation at the group level for recognition. Experiment results on three widely used datasets for group emotion recognition show that our proposed method achieved superior performance in terms of recognition accuracy compared to the state-of-the-art methods.

The performance of all learning-based group emotion recognition (GER) methods depends on the number of labeled samples. Although there are lots of group emotion images available on the Internet, labeling them manually is a labor-intensive and cost-expensive process. For this reason, datasets for GER are usually small in size, which limits the performance of GER. Considering labeling manually is challenging, using limited labeled images and a large number of unlabeled images in the network training is a potential way to improve the performance of GER. In this work, we propose a semi-supervised group emotion recognition framework based on contrastive learning to learn efficient features from both labeled and unlabeled images. In the proposed method, the unlabeled images are used to pretrain the backbone by a contrastive learning method, and the labeled images are used to fine-tune the network. The unlabeled images are then given pseudo-labels by the fine-tuned network and used for further training. In order to alleviate the uncertainty of the given pseudo-labels, we propose a Weight Cross-Entropy Loss (WCE-Loss) to suppress the influence of the samples with unreliable pseudo-labels in the training process. Experiment results on three prominent benchmark datasets for GER show the effectiveness of the proposed framework and its superiority compared with other competitive state-of-the-art methods.

Most Cross-modal hashing methods do not sufficiently exploit the discrimination power of semantic information when learning hash codes, while often involving time-consuming training procedures for large-scale dataset. To tackle these issues, we first formulate the learning of similarity-preserving hash codes in terms of orthogonally rotating the semantic data to hamming space, and then propose a novel Fast Semantic Preserving Hashing (FSePH) approach to large-scale cross-modal retrieval. Specifically, FSePH introduces an orthonormal basis to regress the targeted hash codes of training examples to their corresponding reasonably relaxed class labels, featuring significantly reducing the quantization error. Meanwhile, an effective optimization algorithm is derived for modality-specific projection function learning and an efficient closed-form solution for hash code learning, which are computationally tractable. Extensive experiments have shown that the proposed FSePH approach runs sufficiently fast, and also significantly improves the retrieval performances over the state-of-the-arts.