Rushil Thareja

Recent research has focused on the effectiveness of Virtual Reality (VR) in games as a more immersive method of interaction. However, there is a lack of robust analysis of the physiological effects between VR and flatscreen (FS) gaming. This paper introduces the first systematic comparison and analysis of emotional and physiological responses to commercially available games in VR and FS environments. To elicit these responses, we first selected four games through a pilot study of 6 participants to cover all four quadrants of the valence-arousal space. Using these games, we recorded the physiological activity, including Blood Volume Pulse and Electrodermal Activity, and self-reported emotions of 33 participants in a user study. Our data analysis revealed that VR gaming elicited more pronounced emotions, higher arousal, increased cognitive load and stress, and lower dominance than FS gaming. The Virtual Reality and Flat Screen (VRFS) dataset, containing over 15 hours of multimodal data comparing FS and VR gaming across different games, is also made publicly available for research purposes. Our analysis provides valuable insights for further investigations into the physiological and emotional effects of VR and FS gaming.

The rapid expansion of social media platforms, such as Twitter, has led to an influx of multimodal content frequently exhibiting extreme emotional sentiments. Continuous exposure to such content can deteriorate a user’s mental health. To mitigate this, we introduce Tweet-SentiNet, a robust multimodal framework employing image and text embeddings for nuanced analysis and prediction of sentiment emotions in social media content. Tweet-SentiNet filters extreme sentiments, thereby reducing exposure to potentially damaging emotional content and aiding platform moderators in identifying victims and instigators of hate, as well as potential cases of depression or suicidal tendencies. The proposed method outperforms existing baselines, demonstrating the significance of multimodal features in determining sentiment. To further assess the mental well-being effects of consuming such data, we conduct a user study with 20 participants and present the findings. This paper sheds light on the potential of utilizing multimodal sentiment analysis for mental health monitoring and intervention on social media platforms.