Are Scientists Starting to Sound like ChatGPT? Stylistic Diffusion in Biomedical Abstracts in the LLM Era
Open Access
- 27 July 2026
- journal article
- research article
- Published by MDPI AG in Machine Learning and Knowledge Extraction
- Vol. 8 (8) , 223
Abstract
Large language models are part of everyday scholarly writing, but their influence may not always appear as obvious AI authorship. More often, it may appear through gradual changes in academic style. This study examines whether biomedical abstracts show measurable signs of LLM-associated “same-ification” after the public release of ChatGPT. Instead of classifying individual abstracts as AI-written, our study analyses population-level changes across 28,415 PubMed abstracts published between 2018 and 2025. A strict dictionary of LLM-associated stylistic markers was used alongside a control vocabulary, lexical diversity, semantic similarity, interrupted time-series modelling, journal-level heterogeneity analysis, and a study-specific validation framework called SAME-ML. The findings show a clear post-2022 stylistic shift. Mean strict marker use increased from 4.995 to 11.658 markers per 1000 words, representing a 133.4% rise, while the control vocabulary slightly declined. This increase was strongest in 2024 and 2025 and was mainly driven by markers linked to value framing, contribution signalling, connective phrasing, improvement rhetoric, and problem-solution positioning. SAME-ML further showed that the post-ChatGPT period contained a learnable stylistic signal that remained visible after topic-masked and matched-sample validation. Overall, the findings show a corpus-level rhetorical shift and provide a reproducible approach for tracking stylistic change in scholarly communication over time.Keywords
This publication has 14 references indexed in Scilit:
- Delving into LLM-assisted writing in biomedical publications through excess vocabularyScience Advances, 2025
- Publishers’ and journals’ instructions to authors on use of generative artificial intelligence in academic and scientific publishing: bibliometric analysisBMJ, 2024
- Testing of detection tools for AI-generated textInternational Journal for Educational Integrity, 2023
- Opportunities and challenges for ChatGPT and large language models in biomedicine and healthBriefings in Bioinformatics, 2023
- GPT detectors are biased against non-native English writersPatterns, 2023
- What ChatGPT and generative AI mean for scienceNature, 2023
- ChatGPT: five priorities for researchNature, 2023
- On the Dangers of Stochastic Parrots2021
- stm: An R Package for Structural Topic Models2019
- Interrupted time series regression for the evaluation of public health interventions: a tutorialInternational Journal of Epidemiology, 2016