Artificial Intelligence–Based Co-Facilitator (AICF) for Detecting and Monitoring Group Cohesion Outcomes in Web-Based Cancer Support Groups: Single-Arm Trial Study
Authors:
- Yvonne W LeungYLYvonne W Leung
- Elise WouterlootEWElise Wouterloot
- Achini AdikariAAAchini Adikari
- Jinny HongJHJinny Hong
- Veenaajaa AsokanVAVeenaajaa Asokan
- Lauren DuanLDLauren Duan
- Claire LamCLClaire Lam
- Carlina KimCKCarlina Kim
- Kai P ChanKCKai P Chan
- Daswin De SilvaDSDaswin De Silva
- Lianne TrachtenbergLTLianne Trachtenberg
- Heather RennieHRHeather Rennie
Open Access
- 22 July 2024
- journal article
- research article
- Published by JMIR Publications Inc. in JMIR Cancer
- Vol. 10 (1) , e43070
Abstract
Background: Commonly offered as supportive care, therapist-led online support groups (OSGs) are a cost-effective way to provide support to individuals affected by cancer. One important indicator of a successful OSG session is group cohesion; however, monitoring group cohesion can be challenging due to the lack of nonverbal cues and in-person interactions in text-based OSGs. The Artificial Intelligence–based Co-Facilitator (AICF) was designed to contextually identify therapeutic outcomes from conversations and produce real-time analytics. Objective: The aim of this study was to develop a method to train and evaluate AICF’s capacity to monitor group cohesion. Methods: AICF used a text classification approach to extract the mentions of group cohesion within conversations. A sample of data was annotated by human scorers, which was used as the training data to build the classification model. The annotations were further supported by finding contextually similar group cohesion expressions using word embedding models as well. AICF performance was also compared against the natural language processing software Linguistic Inquiry Word Count (LIWC). Results: AICF was trained on 80,000 messages obtained from Cancer Chat Canada. We tested AICF on 34,048 messages. Human experts scored 6797 (20%) of the messages to evaluate the ability of AICF to classify group cohesion. Results showed that machine learning algorithms combined with human input could detect group cohesion, a clinically meaningful indicator of effective OSGs. After retraining with human input, AICF reached an F1-score of 0.82. AICF performed slightly better at identifying group cohesion compared to LIWC. Conclusions: AICF has the potential to assist therapists by detecting discord in the group amenable to real-time intervention. Overall, AICF presents a unique opportunity to strengthen patient-centered care in web-based settings by attending to individual needs. International Registered Report Identifier (IRRID): RR2-10.2196/21453Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- Making Evidence-Based Psychotherapy More Accessible in CanadaThe Canadian Journal of Psychiatry, 2016
- Self-reference in psychosis and depression: a language marker of illnessPsychological Medicine, 2016
- Me, myself, and I: self-referent word use as an indicator of self-focused attention in relation to depression and anxietyFrontiers in Psychology, 2015
- Linguistic measures of the referential process in psychodynamic treatment: The English and Italian versionsPsychotherapy Research, 2013
- Therapist and relationship factors influencing dropout from individual psychotherapy: A literature reviewPsychotherapy Research, 2013
- Empowering Processes and Outcomes of Participation in Online Support Groups for Patients With Breast Cancer, Arthritis, or FibromyalgiaQualitative Health Research, 2008
- “What concerns me is…” Expression of emotion by advanced cancer patients during outpatient visitsSupportive Care in Cancer, 2007
- Overcoming Depression on the Internet (ODIN) (2): A Randomized Trial of a Self-Help Depression Skills Program With RemindersJournal of Medical Internet Research, 2005
- Linguistic styles: Language use as an individual difference.Journal of Personality and Social Psychology, 1999
- Linguistic predictors of adaptive bereavement.Journal of Personality and Social Psychology, 1997