Generative AI in Academic Writing: A Comparison of DeepSeek, Qwen, ChatGPT, Gemini, Llama, Mistral, and Gemma
Authors:
Top Cited Papers
Preprint
- 15 April 2025
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
These tools, which are free and open-source have made a significant impact on the world. Deepseek and Qwen also have the potential to be used by many researchers and individuals around the world in academic writing and content creation. Therefore, it is important to determine the capacity of these new LLMs to generate high-quality academic content. This study aims to evaluate the academic writing performance of both Qwen 2.5 Max and DeepSeek v3 by comparing these models with popular systems such as ChatGPT, Gemini, Llama, Mistral, and Gemma. In this research, 40 articles on the topics of Digital Twin and Healthcare were used. The method of this study involves using generative AI tools to generate texts based on posed questions and paraphrased abstracts of these 40 articles. Then, the generated texts were evaluated through the plagiarism tool, AI detection tools, word count comparisons, semantic similarity tools and readability assessments. It was observed that plagiarism test result rates were generally higher for the paraphrased abstract texts and lower for the answers generated to the questions, but both were above acceptable levels. In the evaluations made with the AI detection tool, it was determined with high accuracy that all the generated texts were detected as AI-generated. In terms of the generated word count comparison, it was evaluated that all chatbots generated satisfactory amount of content. Semantic similarity tests show that the generated texts have high semantic overlap with the original texts. The readability tests indicated that the generated texts were not sufficiently readable.Keywords
All Related Versions
This publication has 50 references indexed in Scilit:
- Editors’ statement on the responsible use of generative AI technologies in scholarly journal publishingMedicine, Health Care and Philosophy, 2023
- AI-generated research paper fabrication and plagiarism in the scientific communityPatterns, 2023
- ChatGPT listed as author on research papers: many scientists disapproveNature, 2023
- Digital Twin in Healthcare Through the Eyes of the Vitruvian Man2022
- The Case for Digital Twins in Healthcare2022
- A Practical Guide for Plagiarism Detection Through the Coordinated Use of Software and (Human) HardwareJournal of Gastrointestinal Infections, 2022
- The health digital twin: advancing precision cardiovascular medicineNature Reviews Cardiology, 2021
- Is Human Digital Twin possible?Computer Methods and Programs in Biomedicine Update, 2021
- Blockchain-Based Secure Digital Twin Framework for Smart Healthy City2020
- A Proposal to Evolving Towards Digital Twins in Healthcare2020