Classifying human vs. AI text with machine learning and explainable transformer models
Open Access
- 8 December 2025
- journal article
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 43310
Abstract
The rapid proliferation of AI-generated text from models such as ChatGPT-3.5 and ChatGPT-4 has raised critical challenges in verifying content authenticity and ensuring ethical use of language technologies. This study presents a comprehensive framework for distinguishing between human-written and GPT-generated text using a combination of machine learning, sequential deep learning, and transformer-based models. A balanced dataset of 20,000 samples was compiled, incorporating diverse linguistic and topical sources. Traditional algorithms and sequential architectures (LSTM, GRU, BiLSTM, BiGRU) were compared against advanced transformer models, including BERT, DistilBERT, ALBERT, and RoBERTa. Experimental findings revealed that RoBERTa achieved the highest performance (Accuracy = 96.1%), outperforming all baselines. Post-hoc temperature scaling (T = 1.476) improved calibration, while threshold tuning (t = 0.957) enhanced precision for high-stakes applications. McNemar’s test with Holm correction confirmed the statistical significance ( p < 0.05) of RoBERTa’s superiority. Efficiency analysis showed optimal trade-offs between accuracy and latency, and 20% pruning demonstrated sustainability potential. Furthermore, LIME and SHAP explainability analyses highlighted linguistic distinctions between AI-generated and human-authored text, and fine-grained error evaluation confirmed model robustness across text lengths. In conclusion, RoBERTa emerges as a reliable, interpretable, and computationally efficient model for detecting AI-generated content.Keywords
Funding Information
- Asia Pacific University of Technology and Innovation
This publication has 24 references indexed in Scilit:
- Parallelization of LSTM-GRU Architectures for Multivariate Prediction of Stock Prices2024
- Detecting AI Generated Text2024
- GPT-generated Text Detection: Benchmark Dataset and Tensor-based Detection Method2024
- Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and ChallengesACM Computing Surveys, 2024
- Diabetic risk prognosis with tree ensembles integrating feature attribution methodsEvolutionary Intelligence, 2021
- Deep learning: RNNs and LSTM2020
- Edge Computing: Vision and ChallengesIEEE Internet of Things Journal, 2016
- Support Vector Machines with Weighted Powered Kernels for Data ClassificationCommunications in Computer and Information Science, 2012
- Learning from Imbalanced DataIEEE Transactions on Knowledge and Data Engineering, 2009
- Two decades of statistical language modeling: where do we go from here?Proceedings of the IEEE, 2000