Structured Large Language Model Workflows for Motivational Interviewing in Health Behavior Change: Proof-of-Concept Study
Open Access
- 6 July 2026
- journal article
- research article
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 10, e94036
Abstract
Background: Motivational interviewing (MI) is an effective approach for supporting health behaviorchange, but face-to-face delivery is resource-intensive and difficult to scale. Rule-based conversational agents (CAs) can improve access; however, their scripted interactions and limited language flexibility constrain MI delivery. While large language models (LLMs) are increasingly being used for MI coaching, their conversational fidelity and quality compared with human coaches and rule-based CAs remain understudied. Objective: This study aimed to describe the development of an LLM-based CA, Artificially Intelligent Motivational Interviewing (Aimi), orchestrated with structured workflows, and to evaluate its feasibility, conversational fidelity, and user perceptions during MI coaching interactions. Methods: We developed Aimi using structured LLM workflows designed to enhance MI fidelity. We conducted a within-participants study, where 18 adults interacted with (1) Aimi, (2) a novice MI-trained human coach, and (3) a rule-based CA during live text-based role-play coaching sessions. Transcripts were independently evaluated by an MI expert using the Motivational Interviewing Skill Code, Version 2.0 (MISC-2), to assess MI competency and fidelity. Participants completed a user experience questionnaire to provide general feedback and to assess session alliance, dialogue relevance, empathy, engagement, linguistic quality, and perceived motivation to change. Feedback from users was thematically summarized and categorized under strengths and weaknesses for each approach. Results: Aimi achieved fidelity scores comparable to those of the novice human coach and higher than those of the rule-based CA on summary metrics, including higher reflection-to-question ratios (median 0.84, IQR 0.62-0.92 vs 0.62, IQR 0.42-0.74 vs 0.25, IQR 0.17-0.38), more complex reflections (median 66.67%, IQR 46.97%-76.92% vs 50%, IQR 34.38%-61.88% vs 0.00%, IQR 0%-50%), and greater elicitation of client change talk (median 90.83%, IQR 85.89%-100% vs 73.21%, IQR 63.10%-83.19% vs 66.67%, IQR 57.86%-81.94%). User experience ratings showed no significant differences across conditions. User feedback revealed distinct strengths and limitations across the coaching interactions. Participants described Aimi’s interactions as personalized, fluid, and adaptive, though sometimes overly reflective and lengthy. The novice human coach was viewed as empathetic and supportive but slow to respond, whereas the rule-based coach was viewed as efficient and structured yet limited in depth and personalization. Conclusions: This study demonstrates the technical feasibility of structured LLM-workflows for MI coaching and their capacity to maintain conversational fidelity comparable to that of a novice MI-trained human coach. Given the role-play paradigm, single-rater coding, and small convenience sample, these comparative findings should be interpreted as exploratory. Our findings serve as a foundational baseline for the development of scalable behavior change interventions in clinical settings.Keywords
This publication has 50 references indexed in Scilit:
- LLM-based conversational agents for behaviour change support: A randomised controlled trial examining efficacy, safety, and the role of user behaviourInternational Journal of Human-Computer Studies, 2025
- Development and preliminary testing of a secure large language model-based chatbot for brief alcohol counseling in young adultsDrug and Alcohol Dependence, 2025
- Optimal dietary patterns for healthy agingNature Medicine, 2025
- Designing a Large Language Model-Based Coaching Intervention for Lifestyle Behavior Change2024
- Global burden and strength of evidence for 88 risk factors in 204 countries and 811 subnational locations, 1990–2021: a systematic analysis for the Global Burden of Disease Study 2021The Lancet, 2024
- Changing Behavior Using the Transtheoretical Model2020
- Effect of lifestyle interventions on glucose regulation among adults without impaired glucose tolerance or diabetes: A systematic review and meta-analysisDiabetes Research and Clinical Practice, 2017
- Development and validation of a 6-item working alliance questionnaire for repeated administrations during psychotherapy.Psychological Assessment, 2015
- A reusable framework for health counseling dialogue systems based on a behavioral medicine ontologyJournal of Biomedical Informatics, 2011
- The Motivational Interviewing Skill Code: Reliability and a Critical AppraisalBehavioural and Cognitive Psychotherapy, 2005