Mawal A. Mohammed

Multimodal emotion recognition has become an important function for intelligent human-robot interaction, especially in an assistive robotics scenario for individuals with communication disabilities. The paper presents a Hybrid BERT-Vision Transformer (BERT-ViT) system constructed to combine complementary information obtained from natural language and facial expression analysis for enhanced emotional comprehension in assistive response systems. By employing BERT to produce contextual embeddings for language and the Vision Transformer to obtain fine-grained visual features, the proposed BERT-ViT-domain framework performs cross-modal emotion classification in a robust manner. Contrastive learning is applied for the multimodal feature alignment, and then the fusion step concatenates the [CLS] token embeddings from both modalities. The evaluation of the model was conducted with the MELD dataset, and the findings indicate promising performance associated with a mean accuracy on seven emotion classes and performance levels above conventional unimodal and early-fusion counterparts. These results also indicate the model performs relatively well at detecting subtle and complex affective states of concern in relation to ASD support for empathetic interaction, speech impairments, and socially assistive robotics. The modular nature of the model’s architecture provides for seamless integration into assistive platforms, while also supporting feedback and personalization features. The experimental validation includes linguistic profiling, space-time processing of emotions, and performance classification measures of accuracy (98.71%), precision (98.34%), recall (98.00%), and F1-score (98.38%). Also noteworthy is the strong resistance to overfitting with training-validation convergence and low overall error rates (FPR: 1.58%, FNR: 0.98%). This hybrid framework contributes to realizing advanced multimodal emotion recognition and developing a scalable, responsive, and privacy-responsible solution toward emotionally aware assistive robotic applications in sensitive health care and educational contexts.

The rapid proliferation of online information necessitates efficient Recommendation Systems (RSs) to assist users in discovering relevant content. While English-language RSs have received significant attention, research on Arabic RSs remains limited despite the increasing demand for Arabic digital content. This paper addresses the scarcity of Arabic-focused Collaborative Filtering (CF) approaches for RS. Recognizing the wealth of information embedded in user reviews, we propose novel review-based CF approaches tailored for Arabic, aiming to enhance recommendation accuracy for Arab users. Our work comprises three key stages: we first develop a comprehensive Arabic lexicon specifically for the book domain. Secondly, using this lexicon we then propose three distinct sentiment-aware ratings, leveraging sentiment analysis of Arabic reviews to enrich traditional rating predictions. Thirdly, these sentiment-aware ratings are integrated into ten diverse CF algorithms from the Surprise library and a deep autoencoder neural network, covering a spectrum of traditional and modern approaches. Extensive experiments conducted on the Large Arabic Book Reviews (LABR) dataset demonstrate the superior performance of our proposed sentiment-aware ratings compared to baseline methods across all evaluated metrics. Further analysis reveals the importance of appropriate sentiment word extraction methods and lexicon selection for accurate sentiment rating calculation. Finally, this study makes a significant contribution to the field of Arabic CF recommendation systems by providing a comprehensive framework for leveraging user review and underscores the importance of further research in this area.

Collaborative filtering (CF) is one of the most widely utilised approaches in recommendation techniques. It suggests items to users based on the ratings of other users who share their preferences. Thus, one of the aims of CF is to find reliable neighbours. Typically, CF produces a sparse user-item rating matrix, when relying only on the ratings to identify the precise neighbours, resulting in poor performance. User reviews can be essential in overcoming those situations because of the diverse elements available in reviews. The most popular element is aspects, which can provide a fine-grained analysis of users’ behaviours, thus improving personalised recommendations. However, increasing the number of aspects also results in sparsity, therefore may deteriorate the recommendation performance. As a result, clustering of aspects may lessen this sparsity, but it is yet unclear how much this would affect the performance of CF systems. This study proposes a CF approach based on aspect clustering that addresses the above issue in terms of rating prediction. The approach aims to reduce the sparseness in the multi-criteria rating matrix by grouping aspects into clusters based on their semantic similarity, which will be less expensive and require less memory to discover the neighbourhood set. Our approach extracts aspects and represents them using Google’s pre-trained Word2vec model. Then, aspects are organised into clusters using the K-means clustering algorithm. Multi-dimensional Euclidean distance is used as a similarity measure for finding the appropriate neighbours and predicted ratings of unseen items are then made using the kNN algorithm. This study also identifies the number of aspects that significantly impacts CF performance. Experiments are carried out using a real large-scale dataset: the Amazon movie dataset. Evaluation is also performed by comparing CF performance of the proposed approach with three different baseline approaches. Results show that the proposed approach improves CF performance compared to other approaches in terms of three predictive accuracy metrics.