Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues
Top Cited Papers
Open Access
- 16 December 2025
- journal article
- review article
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27, e78238
Abstract
Background: In recent years, artificial intelligence (AI) has driven the rapid development of AI mental health chatbots. Most current reviews investigated the effectiveness of rule-based or retrieval-based chatbots. To date, there is no comprehensive review that systematically synthesizes the effect of generative AI (GenAI) chatbot’s impact on mental health. Objective: This review aims to (1) narratively synthesize existing GenAI mental health chatbots’ technical features, treatment and research designs, and sample characteristics through a systematic review of quantitative studies and (2) quantify the effectiveness and key moderators of these rigorously designed trials on GenAI mental health chatbots through a meta-analysis of only randomized controlled trials (RCTs). Methods: The search strategy includes 11 database searching, backward citation tracking, and a manual ad hoc search to update literature. This thorough literature search, completed in March 2025, returned 5555 records for screening. The systematic review included studies that (1) used generative or hybrid (rule/retrieval-based and generative) AI-based chatbots to deliver interventions and (2) quantitatively measured mental health-related outcomes. The meta-analysis has additional inclusion criteria: (1) studies must be RCTs, (2) must measure negative mental health issues, (3) the comparison group must not have chatbot features, and (4) must provide enough statistics for effect size calculation. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist and registered the protocol retrospectively during the revision process (September 18, 2025). In meta-regression, data were synthesized in R software using a random-effects model. Results: The narrative synthesis of 26 studies revealed that (1) GenAI chatbot interventions mostly took place in non-WEIRD countries (non-Western, Educated, Industrialized, Rich, and Democratic) and (2) there is a lack of studies focusing on young children and older adults. The meta-analysis of 14 RCTs showed a statistically significant effect (effect size [ES]=0.30, P =.047, N =6314, 95% CI 0.004, 0.59, 95% prediction interval [PI] −0.85, 1.67), which means that GenAI chatbots are, on average, effective in reducing negative mental health issues, such as depression, anxiety, among others. We found that social-oriented chatbots (ie, those that mainly provide social interactions) are more effective than task-oriented programs (ie, those that assist with specific tasks). Risk of bias in the nonrandomized studies and RCTs was assessed using Cochrane ROBINS-I (Risk Of Bias In Non-randomised Studies – of Interventions) and RoB2 (revised Cochrane risk-of-bias tool for randomized trials), respectively, indicating a moderate amount of risk. One main limitation of this meta-analysis is the small number of studies (n=14) included. Conclusions: By identifying research gaps, we suggest that future researchers investigate user groups such as adolescents and older adults, outcomes other than depression and anxiety, cultural adaptations in non-WEIRD countries, ways to streamline chatbots in usual care practices, and explore applications in diverse settings. More importantly, we cannot ignore GenAI chatbots’ risks while acknowledging their promise. This review also emphasized several ethical implications.Keywords
This publication has 86 references indexed in Scilit:
- The impact of different conversational generative AI chatbots on EFL learners: An analysis of willingness to communicate, foreign language speaking anxiety, and self-perceived communicative competenceSystem, 2024
- Comparing Patient’s Confidence in Clinical Capabilities in Urology: Large Language Models Versus UrologistsEuropean Urology Open Science, 2024
- The effects of chatbot use on foreign language reading anxiety and reading performance among Chinese secondary school studentsComputers and Education: Artificial Intelligence, 2024
- The therapeutic effectiveness of artificial intelligence-based chatbots in alleviation of depressive and anxiety symptoms in short-course treatments: A systematic review and meta-analysisJournal of Affective Disorders, 2024
- Comparison of Conventional Anesthesia Nurse Education and an Artificial Intelligence Chatbot (ChatGPT) Intervention on Preoperative Anxiety: A Randomized Controlled TrialJournal of PeriAnesthesia Nursing, 2024
- Quantifying the global burden of mental disorders and their economic valueeClinicalMedicine, 2022
- The World Mental Health Report: transforming mental health for allWorld Psychiatry, 2022
- Is chatting with a sophisticated chatbot as good as chatting online or FTF with a stranger?Computers in Human Behavior, 2022
- An overview of the features of chatbots in mental health: A scoping reviewInternational Journal of Medical Informatics, 2019
- Should AI-Based, conversational digital assistants employ social- or task-oriented interaction style? A task-competency and reciprocity perspective for older adultsComputers in Human Behavior, 2019