AI vs. Humans for Online Support: Comparing the Language of Responses from LLMs and Online Communities of Alzheimer’s Disease
Top Cited Papers
Open Access
- 22 June 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
- Vol. 7 (3) , 1-39
Abstract
AI chatbots are increasingly integrated into various sectors, including healthcare. We examine their role in responding to queries related to Alzheimer’s Disease and Related Dementias (AD/ADRD). We obtained real-world queries from AD/ADRD online communities (OCs)—Reddit (r/Alzheimers) and ALZConnected. First, we conducted a small-scale qualitative examination where we prompted ChatGPT, Bard, and Llama-2 with 101 OC posts to generate responses and compared them with OC responses through inductive coding and thematic analysis. We found that although AI can provide emotional and informational support like OCs, they do not engage in deeper conversations, provide references, and share personal experiences. These insights motivated us to conduct a large-scale quantitative examination of comparing AI (GPT) and OC responses (90K) to 13.5K posts, in terms of psycholinguistics, lexico-semantics, and content. AI responses tend to be more verbose, readable, and complex. AI responses exhibited greater empathy, but more formal and analytical language, lacking personal narratives and linguistic diversity. We found that various LLMs, including GPT, Llama, and Mistral, exhibit consistent patterns in responding to AD/ADRD-related queries, underscoring the robustness of our insights across LLMs. Our study sheds light on the potential of AI in digital health and underscores design considerations of AI to complement human interactions.Keywords
This publication has 132 references indexed in Scilit:
- Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated textInternational Journal for Educational Integrity, 2023
- Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research ScenariosJournal of Medical Systems, 2023
- Tracking Fluctuations in Psychological States using Social Media Language: A Case Study of Weekly EmotionEuropean Journal of Personality, 2020
- Syntactic complexity across registers: Investigating (in)formality in second-language writingJournal of English for Academic Purposes, 2020
- Detecting depression and mental illness on social media: an integrative reviewCurrent Opinion in Behavioral Sciences, 2017
- Health-related Internet use among cancer survivors: data from the Health Information National Trends Survey, 2003–2008Journal of Cancer Survivorship, 2011
- Congruence/genuineness.Psychotherapy, 2011
- Social capital, self-esteem, and use of online social network sites: A longitudinal analysisJournal of Applied Developmental Psychology, 2008
- Linguistic predictors of adaptive bereavement.Journal of Personality and Social Psychology, 1997
- A computer readability formula designed for machine scoring.Journal of Applied Psychology, 1975