Semantic Stealth: Adversarial Text Attacks on NLP Using Several Methods
Authors:
- Roopkatha DeyRDRoopkatha Dey
- Aivy DebnathADAivy Debnath
- Arijit MitraAMArijit Mitra
- Jaydip SenJSJaydip Sen
Preprint
Abstract
In various real-world applications such as machine translation, sentiment analysis, and question answering, a pivotal role is played by NLP models, facilitating efficient communication and decision-making processes in domains ranging from healthcare to finance. However, a significant challenge is posed to the robustness of these natural language processing models by text adversarial attacks. These attacks involve the deliberate manipulation of input text to mislead the predictions of the model while maintaining human interpretability. Despite the remarkable performance achieved by state-of-the-art models like BERT in various natural language processing tasks, they are found to remain vulnerable to adversarial perturbations in the input text. In addressing the vulnerability of text classifiers to adversarial attacks, three distinct attack mechanisms are explored in this paper using the victim model BERT: BERT-on-BERT attack, PWWS attack, and Fraud Bargain's Attack (FBA). Leveraging the IMDB, AG News, and SST2 datasets, a thorough comparative analysis is conducted to assess the effectiveness of these attacks on the BERT classifier model. It is revealed by the analysis that PWWS emerges as the most potent adversary, consistently outperforming other methods across multiple evaluation scenarios, thereby emphasizing its efficacy in generating adversarial examples for text classification. Through comprehensive experimentation, the performance of these attacks is assessed and the findings indicate that the PWWS attack outperforms others, demonstrating lower runtime, higher accuracy, and favorable semantic similarity scores. The key insight of this paper lies in the assessment of the relative performances of three prevalent state-of-the-art attack mechanisms.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Efficient Combinatorial Optimization for Word-Level Adversarial Textual AttackIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- Multilingual Universal Sentence Encoder for Semantic Retrieval2020
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- TextBugger: Generating Adversarial Text Against Real-world Applications2019
- Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised LearningIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Black-Box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers2018
- Generating Natural Language Adversarial Examples2018
- Neural Belief Tracker: Data-Driven Dialogue State Tracking2017
- Adversarial Examples for Evaluating Reading Comprehension Systems2017
- A large annotated corpus for learning natural language inference2015