The Potential of AI Chatbots as Diagnostic Tools in Mental Health: Evaluating Exercise Dependence Symptoms
Open Access
- 23 October 2025
- journal article
- research article
- Published by Springer Nature in Journal of Technology in Behavioral Science
- p. 1-9
Abstract
This study aimed to evaluate the effectiveness of AI chatbots (Claude-3.5 Sonnet, ChatGPT-4o, and Gemini-1.5 Pro) in identifying exercise dependence symptoms using a hypothetical case study. To this end, three sport psychologists, each with a minimum of five years of experience, assessed the chatbots' performance in diagnostic assessment competency, implementation of diagnostic criteria, summary quality, terminology use, and overall effectiveness. The results indicated that while all chatbot models successfully identified major symptoms, their performance varied. Specifically, the Claude-3.5 Sonnet model demonstrated superior performance in specific areas, such as providing a clear case summary and using accurate terminology. However, all chatbots exhibited limitations in recognizing symptom severity and distinguishing between primary and secondary dependence. The sport psychologists expressed a willingness to use at least one AI chatbot model as an assistive tool in initial client assessments. These findings highlight the potential value of AI chatbots in mental health assessments. Future research should prioritize the optimization of algorithms and training data through expert collaboration and controlled real-world testing to improve the reliability and practical application of these tools.Keywords
This publication has 50 references indexed in Scilit:
- Use of Artificial Intelligence Chatbots in Interpretation of Pathology ReportsJAMA Network Open, 2024
- The impact of educational chatbot on student learning experienceEducation and Information Technologies, 2023
- How machine learning is used to study addiction in digital healthcare: A systematic reviewInternational Journal of Information Management Data Insights, 2023
- COVID‐19 Clinical Diagnostics and Testing TechnologyPharmacotherapy: The Journal of Human Pharmacology and Drug Therapy, 2020
- An Overview of Chatbot Technology2020
- Prevalence of Risk for Exercise Dependence: A Systematic ReviewSports Medicine, 2018
- The Role of Passion in Exercise Addiction, Exercise Volume, and Exercise Intensity in Long-term ExercisersInternational Journal of Mental Health and Addiction, 2018
- Understanding Exercise DependenceJournal of Contemporary Psychotherapy, 2009
- Exercise dependence: a systematic reviewPsychology of Sport and Exercise, 2002
- Excessive exercise in anorexia nervosa and bulimia nervosa: Relation to eating characteristics and general psychopathologyInternational Journal of Eating Disorders, 2002