Deepfake Text Detection: Limitations and Opportunities
Authors:
- 1 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1613-1630
Abstract
Recent advances in generative models for language have enabled the creation of convincing synthetic text or deepfake text. Prior work has demonstrated the potential for misuse of deepfake text to mislead content consumers. Therefore, deepfake text detection, the task of discriminating between human and machine-generated text, is becoming increasingly critical. Several defenses have been proposed for deepfake text detection. However, we lack a thorough understanding of their real-world applicability. In this paper, we collect deepfake text from 4 online services powered by Transformer-based tools to evaluate the generalization ability of the defenses on content in the wild. We develop several low-cost adversarial attacks, and investigate the robustness of existing defenses against an adaptive attacker. We find that many defenses show significant degradation in performance under our evaluation scenarios compared to their original claimed performance. Our evaluation shows that tapping into the semantic information in the text content is a promising approach for improving the robustness and generalization performance of deepfake text detection schemes.Keywords
All Related Versions
This publication has 48 references indexed in Scilit:
- SQuAD: 100,000+ Questions for Machine Comprehension of Text2016
- Neural Machine Translation of Rare Words with Subword Units2016
- Glove: Global Vectors for Word Representation2014
- Long Short-Term Memory2012
- An Optimal Assessment of Natural Language Student Input Using Word-to-Word Similarity Metrics2012
- Automatic evaluation of machine translation quality using n-gram co-occurrence statistics2002
- The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem SolutionsInternational Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 1998
- A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge.Psychological Review, 1997
- Learning long-term dependencies with gradient descent is difficultIEEE Transactions on Neural Networks, 1994
- A Learning Algorithm for Continually Running Fully Recurrent Neural NetworksNeural Computation, 1989