Kunxia Wang

Emotion Recognition in Conversation (ERC) based on speech signals faces significant challenges due to the contextual dependencies and dynamic interactions between speakers. Although existing Transformer models are capable of capturing long-range dependencies, they still encounter limitations in modeling the global dialogue structure and the emotional representations of individual speakers. We propose a novel model named UCSM-TG, which synergistically integrates Transformer and Gated Recurrent Unit (GRU) to track emotions at utterance, conversation, and speaker levels. The model consists of three core modules, the Utterance-level Emotion Tracking Module (UETM), the Conversation-level Emotion Tracking Module (CETM), and the Speaker Emotion Positioning Module (SEPM). These modules are designed to extract local emotional features, encode global contextual information, and track the emotional states of individual speakers, respectively. Furthermore, We improve Wav2vec 2.0 with the Convolutional Neural Network (CNN) to enhance the feature representation and employ the Focal Loss function to mitigate class imbalance in the datasets. Experimental results demonstrate that UCSM-TG achieves competitive performance, attaining weighted F1-scores of 45.21%, 57.63%, and 68.69%, respectively, on IEMOCAP, MELD, and EEIDB, respectively. Compared with the strong transformer baseline, UCSM-TG achieves weighted F1-score improvements of 3.75%, 1.43% and 1.76%, respectively, on the above datasets, outperforming most state-of-the-art ERC methods.

Theories of Constructed Emotion and Grounded Cognition suggest that our sensorimotor experiences underpin the formation of emotions. This study explores this premise by examining how movements of a computer cursor can reflect moods of participants. We conducted an experiment where participants engaged in a simple choice-reaching task, with their mouse-cursor movements tracked pixel by pixel. Mood assessments were conducted using the PANAS-X scale before and after the task. Through Intersubject Representational Similarity Analysis, we investigated the correlation between the patterns of mouse movements and self-reported moods. Our findings reveal a significant association between negative emotions, such as fear and hostility, and certain movement patterns, e.g., randomness and deviations from a direct path. Furthermore, our machine learning-based Representational Similarity Analysis (ML-RSA) underscores the value of second-order similarity measures, revealing meaningful alignments between sensorimotor behaviors and emotional states across distinct measurement domains. These findings highlight the potential of cursor-tracking as a tool for exploring the interplay between emotion and action.

Mouse-cursor tracking, a new action-based measure of behavior, has emerged as one of the promising applications of affective computing. As facial expressions, gaits, electroencephalogram (EEG), and electrodermal activity (EDA) inform the emotions of computer users, the movement of the computer mouse-cursor reveals when people feel anxious, relaxed, attentive, joyful, and sad. However, the mouse tracking analysis has not previously been subject to systematic investigations of psychometric properties. The choice of motor features, experimental manipulations, and data transformation methods is ad hoc. In this study, we evaluate the impact of psychological factors on mouse-based affective computing and propose simple scoring algorithms that incorporate psychometric features such as the frame of reference, habituation, and measurement error. Our results demonstrate that our new dimensionality reduction method, merged PCA, outperforms conventional procedures, improving prediction performance by about 15 − 30%.

With the development of society, the proportion of the elderly population is increasing year by year, and population aging has become a global trend. As people grow older, the shape and texture features of the face will change to some extent, which increases the difficulty of face recognition. The key point to getting good performance in the elderly expression recognition task is how to capture the facial features effectively. In this paper, we use a Vision Transformer network and apply the Local Mutual Information Maximization (LMIM) technique to it. The correlation between the input map and the high-level hidden representation is evaluated through the optimization of local mutual information. This also allows for the identification of the most pertinent features between the input image and the hidden representation. Finally, we evaluated this method on three datasets, EFED(Self-constructed dataset on the elderly), FER2013, and Seeprettyface as benchmarks, and the experiments show that the experimental results after the addition of local mutual information maximization (LMIM) are improved by about 4% to 9% over the original.

Voice conversion (VC) is a technique that transforms the speech identity of a speaker to that of another while preserving the linguistic content. This technology has various applications such as dubbing for movies, cross-lingual conversion for language learning and speech assistance. However, many VC methods rely on parallel utterances to generate high-quality output. This requirement severely limits the practicality of this technology.We propose an unsupervised many-to-many VC model called MISGAN based on mutual information (MI). By combining adversarial loss and MI minimization loss, our model exhibits superior performance compared to previous VC models and can be employed for a range of voice conversion tasks, including arbitrary many-to-many, cross-lingual, and cross-gender voice conversion. We combine the spectral entropy (SE) of the speech signal with Mel frequency cepstral coefficients (MFCC) and use MI to reduce the correlation between different features, thereby improving the learning efficiency of the model. In addition, our model is fully convolutional and uses a faster and more stable WORLD vocoder for real-time voice conversion.Objective evaluation indicates that our proposed method improves speech quality both globally and locally. In addition, subjective evaluation indicates that our model is superior to Star-GANv2 in naturalness and speaker similarity.