A Comparative Evaluation of Three Large Language Models for Parent‐Centered Questions About Anorexia Nervosa
Open Access
- 20 April 2026
- journal article
- Published by Wiley in International Journal of Eating Disorders
- Vol. 59 (8) , 1764-1775
Abstract
Background: Large language models (LLMs) are increasingly used to obtain health information, including guidance on child and adolescent mental health. In anorexia nervosa (AN), where early recognition and timely intervention are critical, the accuracy of AI‐generated information available to parents may have important clinical implications. This study evaluated the performance of LLMs in responding to parent‐oriented questions about AN. Methods: A comparative model evaluation was conducted using three conversational AI systems: ChatGPT (GPT‐4o), Google Gemini, and DeepSeek. Twenty questions representative of those frequently asked by parents of adolescents with AN were identified through online content exploration and expert review. Each question was submitted using standardized prompts in separate chat sessions. Responses were anonymized and independently evaluated by two board‐certified child and adolescent psychiatrists across three dimensions: quality, usefulness, and reliability. Reproducibility was assessed through repeated queries in separate sessions. Results: All three models demonstrated generally high levels of reliability and overall informational performance. ChatGPT achieved the highest overall accuracy (≈92%) and reproducibility (≈90%), followed by Gemini (≈88% accuracy; ≈85% reproducibility) and DeepSeek (≈86% accuracy; ≈83% reproducibility). Domain‐level analysis showed lower accuracy across models for diagnosis and clinical assessment questions. Qualitative error analysis indicated that the omission of clinically relevant information was the most common limitation, while DeepSeek produced more factual inaccuracies and Gemini more generalized guidance. Conclusions: LLMs may provide broadly accurate preliminary information for parents seeking guidance about AN. However, persistent omissions and domain‐specific variability highlight important limitations. AI‐generated information should therefore be regarded as a complementary resource rather than a replacement for professional clinical guidance.Keywords
This publication has 11 references indexed in Scilit:
- Neurocognitive and Social Cognitive Predictors of Adolescent Major Depressive Disorder: A Machine Learning Classification StudyClinical Child Psychology and Psychiatry, 2026
- Influence of Topic Familiarity and Prompt Specificity on Citation Fabrication in Mental Health Research Using Large Language Models: Experimental StudyJMIR Mental Health, 2025
- Can AI Models like ChatGPT and Gemini Dispel Myths About Children’s and Adolescents’ Mental Health? A Comparative Brief ReportPsychiatry International, 2025
- Comparative Evaluation of Large Language Models in Addressing Autism-Related Information Queries: Insights from ChatGPT, Gemini, and CopilotGazi Medical Journal, 2025
- Performance of large language models ChatGPT and Gemini in child and adolescent psychiatry knowledge assessmentPLOS ONE, 2025
- Testing the capability of generative artificial intelligence for parent and caregiver information seekingFamily Relations, 2025
- The diagnosis and treatment of anorexia nervosa in childhood and adolescenceDeutsches Ärzteblatt international, 2024
- Online Health Information Seeking by Parents for Their Children: Systematic Review and Agenda for Further ResearchJournal of Medical Internet Research, 2020
- Parental caregiver burden and recovery of adolescent anorexia nervosa after multi-family therapyEating Disorders, 2019
- A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability ResearchJournal of Chiropractic Medicine, 2016