Development and Validation of the Artificial Intelligence in Mental Health Scale: Application for AI Mental Health Chatbots
Open Access
- 12 December 2025
- journal article
- research article
- Published by MDPI AG in Healthcare
- Vol. 13 (24) , 3269
Abstract
Background/Objectives: Artificial intelligence (AI)-based chatbots present a viable approach to overcoming several challenges associated with conventional psychotherapy, such as high financial costs, limited access to mental health professionals, and geographical or logistical barriers. Thus, these chatbots are increasingly employed as complementary tools to traditional therapeutic practices in mental health care. Our aim was to develop and validate a scale to measure attitudes toward the use of AI-based chatbots for mental health support, i.e., the Artificial Intelligence in Mental Health Scale (AIMHS). Methods: A multidisciplinary panel of experts assessed the content validity. To confirm face validity, we carried out cognitive interviews and calculated the item-level face validity index. We applied factor analysis to verify the construct structure. We assessed measurement invariance across demographic subgroups. Concurrent validity was evaluated using three valid instruments. Reliability was tested through Cronbach’s alpha, Cohen’s kappa, and the intraclass correlation coefficient. Results: Factor analysis supported a two-factor five-item model. The two factors were technical and personal advantages, and explained 81.28% of the variance. The AIMHS demonstrated adequate concurrent validity, evidenced by statistically significant correlations with Artificial Intelligence Attitude Scale (r = 0.405, p-value < 0.001), Attitudes Towards Artificial Intelligence Scale (acceptance subscale; r = 0.401, p-value < 0.001, fear subscale; r = −0.151, p-value = 0.002), and Short Trust in Automation Scale (r = 0.450, p-value < 0.001). Configural, metric and scalar invariance were supported by our findings. Cronbach’s alpha was 0.798, and intraclass correlation coefficient was 0.938. Cohen’s kappa for the five items ranged from 0.760 to 0.848. Conclusions: The AIMHS is a five-item psychometrically sound and user-friendly instrument capturing two dimensions; technical and personal advantages. Future research should be undertaken to further evaluate the psychometric properties of the AIMHS across diverse populations and contexts.Keywords
All Related Versions
This publication has 72 references indexed in Scilit:
- Artificial Intelligence Attitudes Inventory (AIAI): development and validation using Rasch methodologyCurrent Psychology, 2025
- Chatbots and mental health: Insights into the safety of generative AIJournal of Consumer Psychology, 2023
- University students’ intentions to seek psychological counseling, attitudes toward seeking psychological help, and stigmaSocial Psychiatry and Psychiatric Epidemiology, 2023
- The Threats of Artificial Intelligence Scale (TAI)International Journal of Social Robotics, 2021
- Assessing the Attitude Towards Artificial Intelligence: Introduction of a Short Measure in German, Chinese, and English LanguageKI - Künstliche Intelligenz, 2020
- World Medical Association Declaration of HelsinkiJAMA, 2013
- A Brief Measure for Assessing Generalized Anxiety DisorderArchives of internal medicine (1960), 2006
- Applications of structural equation modeling in marketing and consumer research: A reviewInternational Journal of Research in Marketing, 1996
- An Index of Factorial SimplicityPsychometrika, 1974
- An Inventory for Measuring DepressionArchives of General Psychiatry, 1961