ChatGPT as a mental health advisory service: Comparing evaluations from youth and health professionals
Open Access
- 26 February 2026
- journal article
- Published by SAGE Publications in Digital Health
- Vol. 12
Abstract
Objective: Despite the growing use of artificial intelligence in youth mental health support, little is known about how either young people or health professionals perceive answers to mental health-related inquiries generated by large language models (LLMs). Therefore, we draw on Media Richness Theory to examine how these two user groups perceive the “richness” of text-based communication in this context and whether young people and health professionals differ in their assessment. Methods: A total of 123 young people and 31 health professionals evaluated answers to youth mental health inquiries. Each inquiry had two blinded answers: one generated by ChatGPT (GPT-4) and one written by a health professional. Participants rated the answers for validation, relevance, clarity, and utility and were asked to recommend one or both answers. Open-ended responses elaborating participant choices were also collected. Results: The quantitative findings show that young people and health professionals rated answers from both sources similarly on validation, clarity, and utility. However, young people rated ChatGPT's answers higher for relevance and utility, finding them “richer.” This was supported by qualitative data, where youth preferred ChatGPT's clear and actionable answers. Health professionals showed no strong preference and were more critical, often finding the answers too detailed or lacking empathy. Conclusion: This study is the first to compare youth and professional perspectives on ChatGPT's role in youth mental health advice within a blind evaluation design. We conclude by proposing a hybrid advisory model that combines professional expertise with LLMs to enhance the efficiency, scale, and accessibility of youth mental health advisory services.Keywords
Funding Information
- DAM Foundation ((SDAM_FOR462134))
This publication has 42 references indexed in Scilit:
- Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media ForumJAMA Internal Medicine, 2023
- ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid ConcernsHealthcare, 2023
- Using TikTok for public and youth mental health – A systematic review and content analysisClinical Child Psychology and Psychiatry, 2022
- Deconstructing TikTok Videos on Mental Health: Cross-sectional, Descriptive Content AnalysisJMIR Formative Research, 2022
- Characterizing the dissemination of misinformation on social media in health emergencies: An empirical study based on COVID-19Information Processing & Management, 2021
- Young People’s Use of Digital Health Technologies in the Global North: Narrative ReviewJournal of Medical Internet Research, 2021
- Young People’s Online Help-Seeking and Mental Health Difficulties: Systematic Narrative ReviewJournal of Medical Internet Research, 2019
- An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods StudyJMIR mHealth and uHealth, 2018
- A Systematic Review of Online Youth Mental Health Promotion and Prevention InterventionsJournal of Youth and Adolescence, 2014
- How Adolescents Use Technology for Health Information: Implications for Health Professionals from Focus Group StudiesJournal of Medical Internet Research, 2003