Arabic Sentiment Analysis and Sarcasm Detection Using Probabilistic Projections-Based Variational Switch Transformer
Open Access
- 26 June 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 67865-67881
Abstract
Text classification is a common task in natural language processing (NLP), where the objective is to assign predefined categories or labels to a given text. Detecting sarcasm and classifying sentiment and dialect in NLP has practical applications, including spam detection, topic classification, and sentiment analysis. However, sarcasm and sentimental expressions, such as irony, humor, or criticism, can be difficult to identify through traditional NLP methods due to their implicit nature. To address this, we propose a Modified Switch Transformer (MST) for detecting sarcasm and classifying sentiment and dialect in Arabic text data. Our approach includes two key contributions: Variational Enmesh Expert’s Routing ( V EeR ) and Probabilistic Projections ( P ϕ ). The switch transformer model incorporates probabilistic projections using a Variational Spatial Gated Unit-MLP to enhance the embedding generation mechanism. This updated mechanism introduces a variational aspect, providing dynamic control over the flow of information in the network, in contrast to the simpler embedding generation phase used in the original switch transformer. Moreover, we incorporate Variational Enmesh Expert’s Routing, which utilizes a hierarchical set of Variational experts, where each expert is a small and variational-directed acyclic graph network. The V EeR routing technique allows the network to dynamically choose which path to take at each layer based on the input, using a set of weights learned during training to determine the best route for a given input. Instead of optimizing route paths deterministically, we utilize Variational Inference and model each route as a random variable from a distribution. Our study evaluates the effectiveness of the Modified Switch Transformer (MST) model on the ArSarcasm Dataset, which includes Arabic language data related to sarcasm, dialect, and sentiments. We compare the performance of our proposed model with existing state-of-the-art models in the literature. The results show that the switch transformer outperforms other models in detecting sarcasm and also performs well in classifying sentiment and dialect.Keywords
Funding Information
- Princess Nourah bint Abdulrahman University Researchers Supporting Project (PNURSP2023R407)
- Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia
This publication has 45 references indexed in Scilit:
- A New Text Classifier Based on Random Forests2017
- A Pattern-Based Approach for Sarcasm Detection on TwitterIEEE Access, 2016
- Putting Sarcasm Detection into Context: The Effects of Class Imbalance and Manual Labelling on Supervised Machine Classification of Twitter Conversations2016
- SemEval-2016 Task 7: Determining Sentiment Intensity of English and Arabic Phrases2016
- ASTD: Arabic Sentiment Tweets Dataset2015
- Improved Arabic Dialect Classification with Social Media Data2015
- Sarcastic or Not: Word Embeddings to Predict the Literal or Sarcastic Meaning of Words2015
- Automatic identification of arabic dialects in social media2014
- A study of the effects of preprocessing strategies on sentiment analysis for Arabic textJournal of Information Science, 2014
- Random forests for genomic data analysisGenomics, 2012