Seyed Muhammad Hossein Mousavi

Data scarcity remains a significant challenge in the field of emotion recognition using physiological signals, as acquiring comprehensive and diverse datasets is often prevented by privacy concerns and logistical constraints. This limitation restricts the development and generalization of robust emotion recognition models, making the need for effective synthetic data generation methods more critical. Emotion recognition from physiological signals such as EEG, ECG, and GSR plays a pivotal role in enhancing human-computer interaction and understanding human affective states. Utilizing these signals, this study introduces an innovative approach to synthetic data generation using a Supervised Neural Gas (SNG) network, which has demonstrated noteworthy speed advantages over established models like Conditional VAE, Conditional GAN, diffusion model, and Variational LSTM. The Neural Gas network, known for its adaptability in organizing data based on topological and feature-space proximity, provides a robust framework for generating real-world-like synthetic datasets that preserve the intrinsic patterns of physiological emotion data. Our implementation of the SNG efficiently processes the input data, creating synthetic instances that closely mimic the original data distributions, as demonstrated through comparative accuracy assessments. In experiments, while our approach did not universally outperform all models, it achieved superior performance against most of the evaluated models and offered significant improvements in processing time. These outcomes underscore the potential of using SNG networks for fast, efficient, and effective synthetic data generation in emotion recognition applications.

Automatic emotion recognition has become increasingly important with the rise of AI, especially in fields like healthcare, education, and automotive systems. However, there is a lack of multimodal datasets, particularly involving body motion and physiological signals, which limits progress in the field. To address this, the MVRS dataset is introduced, featuring synchronized recordings from 13 participants aged 12 to 60 exposed to VR based emotional stimuli (relaxation, fear, stress, sadness, joy). Data were collected using eye tracking (via webcam in a VR headset), body motion (Kinect v2), and EMG and GSR signals (Arduino UNO), all timestamp aligned. Participants followed a unified protocol with consent and questionnaires. Features from each modality were extracted, fused using early and late fusion techniques, and evaluated with classifiers to confirm the datasets quality and emotion separability, making MVRS a valuable contribution to multimodal affective computing.

Recognizing human emotions from body motion is a critical challenge in affective computing, particularly in scenarios where facial expressions or speech are unavailable or unreliable. In this study, we propose a novel framework for emotion recognition from skeletal motion data using a graph-based and parallel multi-objective optimization approach. Skeletal motion sequences are represented as graphs, where nodes correspond to joints and edges capture anatomical connections, enabling the preservation of spatial structure and dynamic body patterns crucial for emotional expression. To improve both feature quality and model performance, we employ two evolutionary algorithms in parallel. A Genetic Algorithm (GA) is used to evolve the topology of the motion graphs, optimizing structural characteristics that influence expressiveness. Simultaneously, Particle Swarm Optimization (PSO) is applied to learn optimal joint-level weighting, enhancing the relevance of motion features in the frequency domain. This dual optimization process balances competing objectives, such as accuracy, graph complexity, and interpretability. After extracting graph-theoretic and frequency-domain features from the optimized representations, we train a high-performing Gradient Boosting classifier as a teacher model. To reduce computational cost while retaining predictive power, we distill this knowledge into a lightweight Decision Tree model using a hybrid of soft and hard labels. This knowledge-distilled classifier enables real-time and interpretable emotion recognition with minimal performance degradation. Experiments conducted on a multi-class skeletal emotion dataset show that our method significantly improves recognition accuracy and model efficiency compared to traditional pipelines. The proposed system offers a robust, interpretable, and scalable solution for emotion recognition in human–computer interaction, healthcare, and behavioral analysis applications.

In the domain of emotion recognition using body motion, the primary challenge lies in the scarcity of diverse and generalizable datasets. Automatic emotion recognition uses machine learning and artificial intelligence techniques to recognize a person's emotional state from various data types, such as text, images, sound, and body motion. Body motion poses unique challenges as many factors, such as age, gender, ethnicity, personality, and illness, affect its appearance, leading to a lack of diverse and robust datasets specifically for emotion recognition. To address this, employing Synthetic Data Generation (SDG) methods, such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), offers potential solutions, though these methods are often complex. This research introduces a novel application of the Neural Gas Network (NGN) algorithm for synthesizing body motion data and optimizing diversity and generation speed. By learning the skeletal structure topology, the NGN fits the neurons or gas particles on body joints. Generated gas particles, which form the skeletal structure later on, will be used to synthesize the new body posture. By attaching body postures over frames, the final synthetic body motion appears. We compared our generated dataset against others generated by GANs, VAEs, and another benchmark algorithm, using benchmark metrics such as Fréchet Inception Distance (FID), Diversity, and a few more. Furthermore, we continued evaluation using classification metrics such as accuracy, precision, recall, and a few others. Joint-related features or kinematic parameters were extracted, and the system assessed model performance against unseen data. Our findings demonstrate that the NGN algorithm produces more realistic and emotionally distinct body motion data and does so with more synthesizing speed than existing methods.

Affective computing has recently gained popularity, especially in the field of human-computer interaction systems, where effectively evoking and detecting emotions is of paramount importance to enhance users’ experience. However, several issues are hindering progress in the field. In fact, the complexity of emotions makes it difficult to understand their triggers and control their elicitation. Additionally, effective emotion recognition requires analyzing multiple sensor data, such as facial expressions and physiological signals. These factors combined make it hard to collect high-quality datasets that can be used for research purposes (e.g., development of emotion recognition algorithms). Despite these challenges, Virtual Reality (VR) holds promise as a solution. By providing a controlled and immersive environment, VR enables the replication of real-world emotional experiences and facilitates the tracking of signals indicative of emotional states. However, controlling emotion elicitation remains a challenging task also within VR. This research paper introduces the Magic Xroom, a VR platform designed to enhance control over emotion elicitation by leveraging the theory of flow. This theory establishes a mapping between an individual’s skill levels, task difficulty, and perceived emotions. In the Magic Xroom, the user’s skill level is continuously assessed, and task difficulty is adjusted accordingly to evoke specific emotions. Furthermore, user signals are collected using sensors, and virtual panels are utilized to determine the ground truth emotional states, making the Magic Xroom an ideal platform for collecting extensive datasets. The paper provides detailed implementation information, highlights the main properties of the Magic Xroom, and presents examples of virtual scenarios to illustrate its abilities and capabilities.