A Simple yet Efficient Ensemble Approach for AI-generated Text Detection
Preprint
Abstract
Recent Large Language Models (LLMs) have demonstrated remarkable capabilities in generating text that closely resembles human writing across wide range of styles and genres. However, such capabilities are prone to potential abuse, such as fake news generation, spam email creation, and misuse in academic assignments. Hence, it is essential to build automated approaches capable of distinguishing between artificially generated text and human-authored text. In this paper, we propose a simple yet efficient solution to this problem by ensembling predictions from multiple constituent LLMs. Compared to previous state-of-the-art approaches, which are perplexity-based or uses ensembles with a number of LLMs, our condensed ensembling approach uses only two constituent LLMs to achieve comparable performance. Experiments conducted on four benchmark datasets for generative text classification show performance improvements in the range of 0.5 to 100\% compared to previous state-of-the-art approaches. We also study the influence that the training data from individual LLMs have on model performance. We found that substituting commercially-restrictive Generative Pre-trained Transformer (GPT) data with data generated from other open language models such as Falcon, Large Language Model Meta AI (LLaMA2), and Mosaic Pretrained Transformers (MPT) is a feasible alternative when developing generative text detectors. Furthermore, to demonstrate zero-shot generalization, we experimented with an English essays dataset, and results suggest that our ensembling approach can handle new data effectively.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- MGTBench: Benchmarking Machine-Generated Text Detection2024
- Domain Adaptive Code Completion via Language Models and Decoupled Domain Databases2023
- GPT detectors are biased against non-native English writersPatterns, 2023
- Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language modelsPLOS Digital Health, 2023
- Robo-writers: the rise and risks of language-generating AINature, 2021
- TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation2021
- GLTR: Statistical Detection and Visualization of Generated Text2019
- Hierarchical Neural Story Generation2018