A Comparative Pattern Analysis of Qwen 2.5 and Gemma 3 Text Generation
Authors:
- 29 July 2025
- journal article
- Published by World Scientific and Engineering Academy and Society (WSEAS) in WSEAS TRANSACTIONS ON INFORMATION SCIENCE AND APPLICATIONS
- Vol. 22, 604-615
Abstract
This study examines whether two instruction-tuned language models, Qwen 2.5 (32B) and Gemma 3 (27B), exhibit distinct linguistic patterns for accurate, automated text attribution. We created a dataset of 6,000 LLM-generated text outputs (3,000 per model) to 300 prompts across ten categories for diverse contextual analysis. Afterward, we trained four classifiers: Logistic Regression; Support Vector Machine (SVM); Random Forest; and Gradient Boosting, using Term Frequency-Inverse Document Frequency (TF-IDF) features and various syntactic and stylistic cues. Findings show that TF-IDF features alone are effective for text attribution and that SVM is the most accurate attribution tool, achieving a 99% success rate. At the same time, the ensemble methods of Random Forest and Gradient Boosting were improved by the addition of syntactic and stylistic markers. However, lexical frequency patterns remained the primary predictor, which indicates that simple methods can effectively categorize text. Further analysis also revealed that Qwen 2.5 typically produces structured, formal outputs, while Gemma 3 favors a more expressive, narrative style. Our final results show that all classifiers can effectively identify AI-generated text, which may have future implications for academic integrity, content moderation, and automated plagiarism detection. Considering the constant evolution of Large Language Models (LLMs), better benchmarking methods and additional features are required to precisely attribute AI-generated text across different scenarios.Keywords
This publication has 9 references indexed in Scilit:
- Generative AI in Academic Writing: A Comparison of DeepSeek, Qwen, ChatGPT, Gemini, Llama, Mistral, and GemmaSSRN Electronic Journal, 2025
- The role of generative AI in academic and scientific authorship: an autopoietic perspectiveAI & SOCIETY, 2025
- Can ChatGPT be an author? Generative AI creative writing assistance and perceptions of authorship, creatorship, responsibility, and disclosureSSRN Electronic Journal, 2025
- Deep learning detection method for large language models-generated scientific contentNeural Computing and Applications, 2024
- Comparative Performance of Advanced NLP Models and LLMs in Multilingual Geo-Entity Detection2024
- Detection Vs. Anti-detection: Is Text Generated by AI Detectable?2024
- Testing of detection tools for AI-generated textInternational Journal for Educational Integrity, 2023
- Detection of GPT-4 Generated Text in Higher Education: Combining Academic Judgement and Software to Identify Generative AI Tool MisuseJournal of Academic Ethics, 2023
- Fast and Memory-Efficient TFIDF Calculation for Text Analysis of Large Datasets2021