Reza Safabakhsh

The reliability of a learning model is key to the successful deployment of machine learning in various applications. Creating a robust model, particularly one unaffected by adversarial attacks, requires a comprehensive understanding of the adversarial examples phenomenon. However, it is difficult to describe the phenomenon due to the complicated nature of the problems in machine learning. It has been shown that adversarial training can improve the robustness of the hypothesis. However, this improvement usually comes at the cost of decreased performance on natural samples. Hence, it has been suggested that robustness and accuracy of a hypothesis are at odds with each other. In this article, we put forth the alternative proposal that it is the continuity of a hypothesis that is incompatible with its robustness and accuracy in many of these scenarios. In other words, a continuous function cannot effectively learn the optimal robust hypothesis. We introduce a framework for a rigorous study of harmonic and holomorphic hypotheses in learning theory terms and provide empirical evidence that continuous hypotheses do not perform as well as discontinuous hypotheses in some common machine learning tasks. From a practical point of view, our results suggest that a robust and accurate learning rule would train different continuous hypotheses for different regions of the domain. From a theoretical perspective, our analysis explains the adversarial examples phenomenon in these situations as a conflict between the continuity of a sequence of functions and its uniform convergence to a discontinuous function. Given that many of the contemporary machine learning models are continuous functions, it is important to theoretically study the continuity of robust and accurate classifiers as it is consequential in their construction, analysis, and evaluation. It is important in their construction as discontinuities can lead to artifacts in their approximations. It is necessary to analyze these classifiers as they carry topological information about their domain. It is critical in their evaluation because it reveals the ways that performance metrics, such as the accuracy score, can fail in assessing these classifiers.

Misinformation propagation poses a significant challenge in the digital information era, particularly during crises such as pandemics. Detecting and combating misinformation is crucial for ensuring public health and safety. In this study, we focus on cross-domain misinformation detection for health news by leveraging a Covid-19 dataset to train a model capable of identifying Monkeypox misinformation articles. We propose a novel hybrid model that combines the strengths of both Deep Convolutional Neural Network (DCNN) and Capsule Neural Network (CapsNet), referred to as DCNN-CapsNet. This model reaps the benefits of both DCNN and CapsNet as two different neural network architectures that have been used as classifiers. To enhance the learning process, we employ incremental fine-tuning, which enables our model to begin with dynamic, pre-trained word embeddings that are updated throughout the training phase. This approach ensures that the embeddings remain relevant to the specific task of misinformation detection, adapting to the nuances of the dataset. Additionally, we incorporate margin loss as a critical component of our architecture. Margin loss is designed to improve the model’s ability to learn discriminative features by maximizing intra-cluster similarity and inter-cluster dissimilarity. This loss function addresses the limitations of traditional cross-entropy loss by enforcing a margin between predicted outputs and decision boundaries, thus reducing class overlap and mitigating the risk of overfitting. By ensuring that the learned representations of “real” and “fake” news are sufficiently separated, margin loss enhances the model’s robustness in detecting misinformation across different domains. Our model outperforms several baseline models on a Covid-19 dataset, achieving a 97.34% accuracy and 97.29% F1-score. Furthermore, it surpasses baselines on a Monkeypox dataset, achieving a 96.03% accuracy and 95.75% F1-score. The cross-domain evaluation, training on Covid-19 data, and testing on Monkeypox data, also demonstrates strong performance with an 89.25% accuracy and 94.16% F1-score, exceeding the performance of the same baseline models in the cross-domain setting. These results highlight the effectiveness of our approach for cross-domain misinformation detection and its potential for broader application in combating misinformation across diverse contexts.

In multi-person pose estimation, the coordinates of body joints are estimated to predict the body poses of several people in an image. Although significant progress has been made in this area in recent years, the solutions still suffer from certain weaknesses. One weakness is the detection of a larger number of poses than the actual number of human instances in the image. To face this problem, the final poses are often selected by thresholding the detected human scores. This, however, is not optimal because a pose with a lower priority may be assigned a higher score and selected as the final pose. In this paper, we introduce a problem, explain the reason, and present a pose scoring model as a solution. This model predicts the confidence score for the pose based on the human image, the location and the confidence score of the joints and the structure between the joints of the body. We formulate the problem using Conditional Random Fields and introduce suitable potential functions. To infer the best pose score, we design a neural network whose inputs are a human image and an initial pose. To train the model, a lot of training data and corresponding labels should be provided. For this purpose, we introduce Monte Carlo and GAN-based data augmentation methods, and a scoring function to train the model. Also, we investigate the effect of the pose scores on the reliability and accuracy of multi-person pose estimation models. The proposed idea is implemented as a deep neural network and can be easily applied to other multi-person pose estimation models as a model-agnostic method. In addition to the usual pose estimation evaluation metrics, we consider the number of poses to compare the models in terms of reliability. Experiments show that the performance of the proposed model is higher than the state-of-the-art models. Conducting further research can help to simplify and reduce the complexity of the model.

Automatic video description necessitates generating natural language statements that encapsulate the actions, events, and objects within a video. An essential human capability in describing videos is to vary the level of detail, a feature that existing automatic video description methods, which typically generate single, fixed-level detail sentences, often overlook. This work delves into video descriptions of manipulation actions, where varying levels of detail are crucial to conveying information about the hierarchical structure of actions, also pertinent to contemporary robot learning techniques. We initially propose two frameworks: a hybrid statistical model and an end-to-end approach. The hybrid method, requiring significantly less data, statistically models uncertainties within video clips. Conversely, the end-to-end method, more data-intensive, establishes a direct link between the visual encoder and the language decoder, bypassing any statistical processing. Furthermore, we introduce an Integrated Method, aiming to amalgamate the benefits of both the hybrid statistical and end-to-end approaches, enhancing the adaptability and depth of video descriptions across different data availability scenarios. All three frameworks utilize LSTM stacks to facilitate description granularity, allowing videos to be depicted through either succinct single sentences or elaborate multi-sentence narratives. Quantitative results demonstrate that these methods produce more realistic descriptions than other competing approaches.

Nuanced understanding and the generation of detailed descriptive content for (bimanual) manipulation actions in videos is important for disciplines such as robotics, human-computer interaction, and video content analysis. This study describes a novel method, integrating graph based modeling with layered hierarchical attention mechanisms, resulting in higher precision and better comprehensiveness of video descriptions. To achieve this, we encode, first, the spatio-temporal inter dependencies between objects and actions with scene graphs and we combine this, in a second step, with a novel 3-level architecture creating a hierarchical attention mechanism using Graph Attention Networks (GATs). The 3-level GAT architecture allows recognizing local, but also global contextual elements. This way several descriptions with different semantic complexity can be generated in parallel for the same video clip, enhancing the discriminative accuracy of action recognition and action description. The performance of our approach is empirically tested using several 2D and 3D datasets. By comparing our method to the state of the art we consistently obtain better performance concerning accuracy, precision, and contextual relevance when evaluating action recognition as well as description generation. In a large set of ablation experiments we also assess the role of the different components of our model. With our multi-level approach the system obtains different semantic description depths, often observed in descriptions made by different people, too. Furthermore, better insight into bimanual hand-object interactions as achieved by our model may portend advancements in the field of robotics, enabling the emulation of intricate human actions with heightened precision.

Semantic facial attribute editing using pre-trained Generative Adversarial Networks (GANs) has attracted a great deal of attention and effort from researchers in recent years. Due to the high quality of face images generated by StyleGANs, much work has focused on the StyleGANs' latent space and the proposed methods for facial image editing. Although these methods have achieved satisfying results for manipulating user-intended attributes, they have not fulfilled the goal of preserving the identity, which is an important challenge. We present ID-Style, a new architecture capable of addressing the problem of identity loss during attribute manipulation. The key components of ID-Style include Learnable Global Direction (LGD), which finds a shared and semi-sparse direction for each attribute, and an Instance-Aware Intensity Predictor (IAIP) network, which finetunes the global direction according to the input instance. Furthermore, we introduce two losses during training to enforce the LGD to find semi-sparse semantic directions, which along with the IAIP, preserve the identity of the input instance. Despite reducing the size of the network by roughly 95% as compared to similar state-of-the-art works, it outperforms baselines by 10% and 7% in Identity preserving metric (FRS) and average accuracy of manipulation (mACC), respectively.

News consumption has significantly increased with the growing popularity and use of web-based forums and social media. This sets the stage for misinforming and confusing people. To help reduce the impact of misinformation on users' potential health-related decisions and other intents, it is desired to have machine learning models to detect and combat fake news automatically. This paper proposes a novel transformer-based model using Capsule neural Networks(CapsNet) called X-CapsNet. This model includes a CapsNet with dynamic routing algorithm paralyzed with a size-based classifier for detecting short and long fake news statements. We use two size-based classifiers, a Deep Convolutional Neural Network (DCNN) for detecting long fake news statements and a Multi-Layer Perceptron (MLP) for detecting short news statements. To resolve the problem of representing short news statements, we use indirect features of news created by concatenating the vector of news speaker profiles and a vector of polarity, sentiment, and counting words of news statements. For evaluating the proposed architecture, we use the Covid-19 and the Liar datasets. The results in terms of the F1-score for the Covid-19 dataset and accuracy for the Liar dataset show that models perform better than the state-of-the-art baselines.

We propose a neural network-based approach to assist positioning in an integrated Inertial Navigation System/Navigation Satellite System (INS/GNSS) during GNSS interruptions. One can aid a navigation system during GNSS outages by substituting the GNSS measurements with a well-trained Neural Network (NN). Outputs of this NN can then be used in a Kalman filtering scheme to acquire the best consistent and concise estimates according to the INS measurements. Since this problem has inherent spatial and temporal aspects, the proposed NN should account for both aspects simultaneously. Convolutional Long Short-term Memory (CLSTM) structure is an excellent candidate to satisfy the aforementioned requirement. The designed CLSTM architecture uses the angular rates and specific forces measured by INS to output the pseudo GNSS position increment covering for the lost GNSS signal. An attention mechanism is added to the final layer of the CLSTM to counter the gradient vanishing problem in long time-series prediction (ACLSTM). A field test utilizing a fixed-wing Unmanned Aerial Vehicle (UAV) is arranged to evaluate the proposed architecture's performance. The field test result shows a significant 3-dimensional positioning improvement during GNSS outages.