M. Omair Shafiq

Community detection is crucial for applications like targeted marketing and recommendation systems. Traditional methods rely on network structure, and embedding-based models integrate semantic information. However, there is a challenge when a model leverages local and global information from complex structures like social networks. Graph Neural Networks (GNNs) and Transformers have shown superior performance in capturing local and global relationships. In this paper, We propose Graph Integrated Transformer for Community Detection (GIT-CD), a hybrid model combining GNNs and Transformer-based attention mechanisms to enhance community detection in social networks. Specifically, the GNN module captures local graph structures, while the Transformer module models long-range dependencies. A self-optimizing clustering module refines community assignments using K-Means, silhouette loss, and KL divergence minimization. Experimental results on benchmark datasets show that GIT-CD outperforms state-of-the-art models, making it a robust approach for detecting meaningful communities in complex social networks.

While large language models exhibit broad capabilities, they often require supplementation for optimal performance on domain-specific tasks. To address this, we built TriRAG, an innovative enhancement to traditional Retrieval-Augmented Generation. This paper presents an extended version of our earlier TriRAG work. TriRAG integrates a structured knowledge graph composed of semantic triples derived from text, significantly improving the performance of LLMs on multiple-choice question-answering tasks. Our method dynamically converts relevant text into triples, embeds them into vectors, and retrieves the most useful triples for a given question by calculating vector similarities. By having the triple-based approach instead of the conventional text-based retrieval approach, TriRAG enables more precise and efficient information retrieval. This directly enhances the accuracy of LLM-generated responses in multiple-choice question-answering tasks. We evaluate TriRAG using the Textbook Question Answering dataset, demonstrating consistent improvements over traditional RAG methods across leading LLMs, including Gemma, Llama, and ChatGPT variants. Experimental results and ablation studies confirm that our triple-based system enhances both retrieval accuracy and processing efficiency, leading to better overall model performance.

Machine learning models have made many decision support systems to be faster, more accurate and more efficient. However, applications of machine learning in network security face more disproportionate threat of active adversarial attacks compared to other domains. This is because machine learning applications in network security such as malware detection, intrusion detection, and spam filtering are by themselves adversarial in nature. In what could be considered an arm's race between attackers and defenders, adversaries constantly probe machine learning systems with inputs which are explicitly designed to bypass the system and induce a wrong prediction. In this survey, we first provide a taxonomy of machine learning techniques, tasks, and depth. We then introduce a classification of machine learning in network security applications. Next, we examine various adversarial attacks against machine learning in network security and introduce two classification approaches for adversarial attacks in network security. First, we classify adversarial attacks in network security based on a taxonomy of network security applications. Secondly, we categorize adversarial attacks in network security into a problem space vs. feature space dimensional classification model. We then analyze the various defenses against adversarial attacks on machine learning-based network security applications. We conclude by introducing an adversarial risk grid map and evaluate several existing adversarial attacks against machine learning in network security using the risk grid map. We also identify where each attack classification resides within the adversarial risk grid map.

The transformer model is a famous natural language processing model proposed by Google in 2017. Now, with the extensive development of deep learning, many natural language processing tasks can be solved by deep learning methods. After the BERT model was proposed, many pre-trained models such as the XLNet model, the RoBERTa model, and the ALBERT model were also proposed in the research community. These models perform very well in various natural language processing tasks. In this paper, we describe and compare these well-known models. In addition, we also apply several types of existing and well-known models which are the BERT model, the XLNet model, the RoBERTa model, the GPT2 model, and the ALBERT model to different existing and well-known natural language processing tasks, and analyze each model based on their performance. There are a few papers that comprehensively compare various transformer models. In our paper, we use six types of well-known tasks, such as sentiment analysis, question answering, text generation, text summarization, name entity recognition, and topic modeling tasks to compare the performance of various transformer models. In addition, using the existing models, we also propose ensemble learning models for the different natural language processing tasks. The results show that our ensemble learning models perform better than a single classifier on specific tasks.Graphical Abstract

Question-answering systems are widely used in var-ious productions, and the use of knowledge graphs has been proven to improve their performance. However, current question-answering systems utilizing knowledge graphs suffer from the long tail effect, where high-degree nodes in the knowledge graph are favored by the model. This can lead to a bias in the model towards rare or complex answers, negatively impacting the performance of the question-answering system on questions with uncommon answers. In this paper, we propose a method, ConKGP, to improve the performance of the model QA-GNN. QA-GNN is a state-of-the-art question-answering method that combines language models and knowledge graphs. For the original model structure, we combine contrastive learning to obtain better representation and evenly distribute representation in the data space. Furthermore, we add perturbations to the input of the decoder to enhance the robustness and generalization ability of the model. Our proposed method achieves better results on CommonsenseQA and MedQA-USMLE datasets. The average improvements over QA-GNN are 4.3% and 3.6% respectively. Our experiments show that ConKGP can effectively solve the long tail effect and improve the performance of QA-GNN in processing uncommon answers.

Convolutional neural networks (CNNs) have shown promising results and have outperformed classical machine learning techniques in tasks such as image classification and object recognition. Their human-brain like structure enabled them to learn sophisticated features while passing images through their layers. However, their lack of explainability led to the demand for interpretations to justify their predictions. Research on Explainable AI or XAI has gained momentum to provide knowledge and insights into neural networks. This study summarizes the literature to gain more understanding of explainability in CNNs (i.e., Explainable Convolutional Neural Networks). We classify models that made efforts to improve the CNNs interpretation. We present and discuss taxonomies for XAI models that modify CNN architecture, simplify CNN representations, analyze feature relevance, and visualize interpretations. We review various metrics used to evaluate XAI interpretations. In addition, we discuss the applications and tasks of XAI models. This focused and extensive survey develops a perspective on this area by addressing suggestions for overcoming XAI interpretation challenges, like models’ generalization, unifying evaluation criteria, building robust models, and providing interpretations with semantic descriptions. Our taxonomy can be a reference to motivate future research in interpreting neural networks.

Travel Time Prediction (TTP) has become an essential service that people use in daily commutes. With the precise TTP, individuals, logistic companies, and transport authorities can better manage their activities and operations. This paper presents a novel Hybridized Deep Feature Space (HDFS) based TTP ensemble model (HDFS-TTP) for accurate travel time prediction. In the first step, extensive endogenous and exogenous data sources are augmented with traffic data obtained using sensors. Next, we used Principal Component Analysis (PCA) and Deep Stacked Auto-Encoder (DSAE) for feature reduction. We generated feature spaces of deep learning models, namely Convolutional Neural Network (CNN), Multi-Layer Perceptron (MLP), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU), and fed them to a model based on Support Vector Regressor (SVR) for predicting travel times. Two best-performing models are selected, and their feature spaces are hybridized to boost feature space. On this boosted feature space, we employed SVR for final prediction. Our proposed HDFS-TTP ensemble can learn complex non-linearities in traffic data with the varying architectural design. The performance of our proposed HDFS-TTP ensemble using hybridized and boosted feature spaces showed significant improvement in test data in terms of Root Mean Square Error (62.27± 1.58), Mean Absolute Error (13.38± 1.09), Maximum Absolute Error (104.66± 2.77), Mean Absolute Percentage Error (2.50± 0.03), and Coefficient of determination (0.99714± 0.00044).

Performing sentiment analysis with high accuracy using machine-learning techniques requires a large quantity of training data. However, getting access to such a large quantity of labeled data for specific domains can be expensive and time-consuming. These warrant developing more efficient techniques that can perform sentiment analysis with high accuracy with a few labeled training data. In this paper, we aim to address this problem with our proposed novel sentiment analysis technique, named GAN-BElectra. With rigorous experiments, we demonstrate that GAN-BElectra outperforms its baseline technique in terms of multiclass sentiment analysis accuracy with a few labeled data while maintaining an architecture with reduced complexity compared to its predecessor.