When Artificial Intelligence Speaks For The Obstetrician: Multilingual Accuracy On Real Patient Questions
- 28 August 2026
- journal article
- Published by Bandirma Onyedi Eylul Universitesi in Hippocrates Medical Journal
- Vol. 6 (2) , 61-69
Abstract
BackgroundThe use of generative large language models (LLMs) in healthcare is rapidly increasing, offering easier access to medical information. However, comprehensive data on their multilingual accuracy and the reliability of cited scientific references remain limited. This study aimed to compare the response accuracy and reference quality of three free LLMs (ChatGPT, Google Gemini, DeepSeek) in Turkish and English using common pregnancy-related questions.MethodsIn this comparative observational study, 14 frequently asked pregnancy questions were posed to each LLM in Turkish and English, requesting responses supported by up-to-date scientific web sources. Answers were evaluated blindly by obstetricians and gynaecologists for accuracy. References were independently assessed for reliability, scientific validity, and accessibility. Statistical analyses were performed.ResultsLanguage and model infrastructure significantly influenced performance. Google Gemini and DeepSeek provided more accurate responses in English than in Turkish (pKeywords
All Related Versions
Funding Information
- No financial support was received from any institution.
This publication has 26 references indexed in Scilit:
- Artificial intelligence in obstetrics and gynecology: Evaluating ChatGPT and Google Gemini in answering patient questionsInternational Journal of Gynecology & Obstetrics, 2025
- Evaluation of the Performance of Three Large Language Models in Clinical Decision Support: A Comparative Study Based on Actual CasesJournal of Medical Systems, 2025
- Evaluation of the performance of GPT-3.5 and GPT-4 on the Polish Medical Final ExaminationScientific Reports, 2023
- How accurate are the references generated by ChatGPT in internal medicine?Internal and Emergency Medicine, 2023
- Performance of ChatGPT in French language Parcours d'Accès Spécifique Santé test and in OBGYNInternational Journal of Gynecology & Obstetrics, 2023
- Artificial Intelligence in Clinical DiagnosisJAMA, 2023
- Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media ForumJAMA Internal Medicine, 2023
- ChatGPT outscored human candidates in a virtual objective structured clinical examination in obstetrics and gynecologyAmerican Journal of Obstetrics and Gynecology, 2023
- ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scopeInternet of Things and Cyber-Physical Systems, 2023
- AI in health and medicineNature Medicine, 2022