Delta-Marie Lewis
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Delta-Marie Lewis has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Yun Xian Ho
Dimagi, Inc, Cambridge, MA, United States
5 shared publications - Alyssa J. Martinez
Duke Global Health Institute Duke University Durham North Carolina United States of America
2 shared publications - Anubhuti Poudyal
Department of Sociomedical Sciences, Columbia University, New York, New York, USA
2 shared publications - Dallas Swendeman
Department of Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine, University of California, Los Angeles, USA
2 shared publications - David Warren
Dimagi, Inc, Cambridge, MA, United States
2 shared publications - Gladys Pachas
Center for Addiction Medicine, Department of Psychiatry, Massachusetts General Hospital/Harvard Medical School, Boston, MA, United States
2 shared publications
Latest Publications
- Open Access
- 20 May 2026
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 10, e84683
Development, Feasibility, Acceptability, and Usability of an Artificial Intelligence–Powered Chatbot (Suzy) to Support Patients in Substance Use Disorder Recovery: Multiphase Study
Background: Substance use disorder (SUD) remains a major public health crisis in the United States, with significant challenges in treatment access, retention, and workforce capacity. SUD care teams, including addiction medicine physicians and peer recovery coaches (PRCs), support patients receiving SUD treatment but face heavy workloads and burnout. Artificial intelligence (AI) innovations, particularly large language model (LLM)–based chatbots, may extend PRC support and provide patients with on-demand recovery support between clinic visits and PRC contacts. However, evidence on their development, feasibility, acceptability, and usability in addiction services remains limited. Objective: This study describes the development, feasibility, acceptability, and usability of an AI-powered health coaching chatbot (Suzy) designed to support patients in SUD recovery. Methods: A total of 2 clinicians, 5 researchers, and 2 technology developers led a small, multiphase pilot study. In the formative phase, they conducted focus groups and qualitative in-depth interviews with 12 health care professionals and 8 patients with substance use histories to specify chatbot functions and develop a rule-based chatbot. In phase 2, they conducted usability testing of the rule-based chatbot with 8 patients who reported substance use and completed standardized tasks, surveys, and qualitative interviews. Measures included the System Usability Scale (SUS), Net Promoter Score (NPS), and Single Ease of Use Question (SEQ). In phase 3, they developed an LLM-based chatbot co-designed and fine-tuned with PRCs and other SUD experts. Results: Rule-based chatbot functions included craving management, appointment reminders, resource referrals, care team contacts, and goal setting. Usability task testing supported feasibility. In this small pilot sample, quantitative and qualitative feedback indicated acceptability and usability, with an average SUS score of 93 (benchmark 68), an NPS of 63 (benchmark 35), and a mean SEQ score of 6.5/7. Patients valued Suzy’s approachable, nonjudgmental language and features that promoted accountability, self-monitoring, and 24/7 availability, while emphasizing that chatbots should supplement but not replace human support. The LLM-based chatbot development emphasized information accuracy, safety escalation protocols to mitigate risks of inappropriate chatbot responses, human-in-the-loop features, and expanded conversational flexibility and personal tailoring. Conclusions: In this pilot study, a rule-based chatbot designed to support SUD care demonstrated feasibility, usability, and acceptability. LLM-based chatbot development required more robust safety and emergency reporting features, while offering more patient-responsive conversational functions. By providing on-demand coaching, referrals, and reminders, Suzy may extend the reach of care teams, alleviate provider burden, and enhance patient engagement. Additional work is needed to understand how to best integrate Suzy into patients’ recovery journeys to ensure human support remains accessible and prioritized. LLM evaluation was based on expert testing and safety review. Clinical effectiveness, including the impact on substance use, was not evaluated. Next steps include evaluating the LLM chatbot in real-world settings with larger samples and assessing its efficacy in reducing substance use.
- 20 May 2026
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 10, e84683
Open AccessDevelopment, Feasibility, Acceptability, and Usability of an Artificial Intelligence–Powered Chatbot (Suzy) to Support Patients in Substance Use Disorder Recovery: Multiphase Study
Background: Substance use disorder (SUD) remains a major public health crisis in the United States, with significant challenges in treatment access, retention, and workforce capacity. SUD care teams, including addiction medicine physicians and peer recovery coaches (PRCs), support patients receiving SUD treatment but face heavy workloads and burnout. Artificial intelligence (AI) innovations, particularly large language model (LLM)–based chatbots, may extend PRC support and provide patients with on-demand recovery support between clinic visits and PRC contacts. However, evidence on their development, feasibility, acceptability, and usability in addiction services remains limited. Objective: This study describes the development, feasibility, acceptability, and usability of an AI-powered health coaching chatbot (Suzy) designed to support patients in SUD recovery. Methods: A total of 2 clinicians, 5 researchers, and 2 technology developers led a small, multiphase pilot study. In the formative phase, they conducted focus groups and qualitative in-depth interviews with 12 health care professionals and 8 patients with substance use histories to specify chatbot functions and develop a rule-based chatbot. In phase 2, they conducted usability testing of the rule-based chatbot with 8 patients who reported substance use and completed standardized tasks, surveys, and qualitative interviews. Measures included the System Usability Scale (SUS), Net Promoter Score (NPS), and Single Ease of Use Question (SEQ). In phase 3, they developed an LLM-based chatbot co-designed and fine-tuned with PRCs and other SUD experts. Results: Rule-based chatbot functions included craving management, appointment reminders, resource referrals, care team contacts, and goal setting. Usability task testing supported feasibility. In this small pilot sample, quantitative and qualitative feedback indicated acceptability and usability, with an average SUS score of 93 (benchmark 68), an NPS of 63 (benchmark 35), and a mean SEQ score of 6.5/7. Patients valued Suzy’s approachable, nonjudgmental language and features that promoted accountability, self-monitoring, and 24/7 availability, while emphasizing that chatbots should supplement but not replace human support. The LLM-based chatbot development emphasized information accuracy, safety escalation protocols to mitigate risks of inappropriate chatbot responses, human-in-the-loop features, and expanded conversational flexibility and personal tailoring. Conclusions: In this pilot study, a rule-based chatbot designed to support SUD care demonstrated feasibility, usability, and acceptability. LLM-based chatbot development required more robust safety and emergency reporting features, while offering more patient-responsive conversational functions. By providing on-demand coaching, referrals, and reminders, Suzy may extend the reach of care teams, alleviate provider burden, and enhance patient engagement. Additional work is needed to understand how to best integrate Suzy into patients’ recovery journeys to ensure human support remains accessible and prioritized. LLM evaluation was based on expert testing and safety review. Clinical effectiveness, including the impact on substance use, was not evaluated. Next steps include evaluating the LLM chatbot in real-world settings with larger samples and assessing its efficacy in reducing substance use.
- Open Access
- 7 October 2025
- Published by JMIR Publications Inc. in JMIR Preprints
Development, Feasibility, Acceptability, and Usability of an Artificial Intelligence–Powered Chatbot (Suzy) to Support Patients in Substance Use Disorder Recovery: Multiphase Study (Preprint)
BACKGROUND Substance use disorder (SUD) remains a major public health crisis in the United States, with significant challenges in treatment access, retention, and workforce capacity. SUD care teams, including addiction medicine physicians and peer recovery coaches (PRCs), support patients receiving SUD treatment but face heavy workloads and burnout. Artificial intelligence (AI) innovations, particularly large language model (LLM)–based chatbots, may extend PRC support and provide patients with on-demand recovery support between clinic visits and PRC contacts. However, evidence on their development, feasibility, acceptability, and usability in addiction services remains limited. OBJECTIVE This study describes the development, feasibility, acceptability, and usability of an AI-powered health coaching chatbot (Suzy) designed to support patients in SUD recovery. METHODS A total of 2 clinicians, 5 researchers, and 2 technology developers led a small, multiphase pilot study. In the formative phase, they conducted focus groups and qualitative in-depth interviews with 12 health care professionals and 8 patients with substance use histories to specify chatbot functions and develop a rule-based chatbot. In phase 2, they conducted usability testing of the rule-based chatbot with 8 patients who reported substance use and completed standardized tasks, surveys, and qualitative interviews. Measures included the System Usability Scale (SUS), Net Promoter Score (NPS), and Single Ease of Use Question (SEQ). In phase 3, they developed an LLM-based chatbot co-designed and fine-tuned with PRCs and other SUD experts. RESULTS Rule-based chatbot functions included craving management, appointment reminders, resource referrals, care team contacts, and goal setting. Usability task testing supported feasibility. In this small pilot sample, quantitative and qualitative feedback indicated acceptability and usability, with an average SUS score of 93 (benchmark 68), an NPS of 63 (benchmark 35), and a mean SEQ score of 6.5/7. Patients valued Suzy’s approachable, nonjudgmental language and features that promoted accountability, self-monitoring, and 24/7 availability, while emphasizing that chatbots should supplement but not replace human support. The LLM-based chatbot development emphasized information accuracy, safety escalation protocols to mitigate risks of inappropriate chatbot responses, human-in-the-loop features, and expanded conversational flexibility and personal tailoring. CONCLUSIONS In this pilot study, a rule-based chatbot designed to support SUD care demonstrated feasibility, usability, and acceptability. LLM-based chatbot development required more robust safety and emergency reporting features, while offering more patient-responsive conversational functions. By providing on-demand coaching, referrals, and reminders, Suzy may extend the reach of care teams, alleviate provider burden, and enhance patient engagement. Additional work is needed to understand how to best integrate Suzy into patients’ recovery journeys to ensure human support remains accessible and prioritized. LLM evaluation was based on expert testing and safety review. Clinical effectiveness, including the impact on substance use, was not evaluated. Next steps include evaluating the LLM chatbot in real-world settings with larger samples and assessing its efficacy in reducing substance use.
- 7 October 2025
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessDevelopment, Feasibility, Acceptability, and Usability of an Artificial Intelligence–Powered Chatbot (Suzy) to Support Patients in Substance Use Disorder Recovery: Multiphase Study (Preprint)
BACKGROUND Substance use disorder (SUD) remains a major public health crisis in the United States, with significant challenges in treatment access, retention, and workforce capacity. SUD care teams, including addiction medicine physicians and peer recovery coaches (PRCs), support patients receiving SUD treatment but face heavy workloads and burnout. Artificial intelligence (AI) innovations, particularly large language model (LLM)–based chatbots, may extend PRC support and provide patients with on-demand recovery support between clinic visits and PRC contacts. However, evidence on their development, feasibility, acceptability, and usability in addiction services remains limited. OBJECTIVE This study describes the development, feasibility, acceptability, and usability of an AI-powered health coaching chatbot (Suzy) designed to support patients in SUD recovery. METHODS A total of 2 clinicians, 5 researchers, and 2 technology developers led a small, multiphase pilot study. In the formative phase, they conducted focus groups and qualitative in-depth interviews with 12 health care professionals and 8 patients with substance use histories to specify chatbot functions and develop a rule-based chatbot. In phase 2, they conducted usability testing of the rule-based chatbot with 8 patients who reported substance use and completed standardized tasks, surveys, and qualitative interviews. Measures included the System Usability Scale (SUS), Net Promoter Score (NPS), and Single Ease of Use Question (SEQ). In phase 3, they developed an LLM-based chatbot co-designed and fine-tuned with PRCs and other SUD experts. RESULTS Rule-based chatbot functions included craving management, appointment reminders, resource referrals, care team contacts, and goal setting. Usability task testing supported feasibility. In this small pilot sample, quantitative and qualitative feedback indicated acceptability and usability, with an average SUS score of 93 (benchmark 68), an NPS of 63 (benchmark 35), and a mean SEQ score of 6.5/7. Patients valued Suzy’s approachable, nonjudgmental language and features that promoted accountability, self-monitoring, and 24/7 availability, while emphasizing that chatbots should supplement but not replace human support. The LLM-based chatbot development emphasized information accuracy, safety escalation protocols to mitigate risks of inappropriate chatbot responses, human-in-the-loop features, and expanded conversational flexibility and personal tailoring. CONCLUSIONS In this pilot study, a rule-based chatbot designed to support SUD care demonstrated feasibility, usability, and acceptability. LLM-based chatbot development required more robust safety and emergency reporting features, while offering more patient-responsive conversational functions. By providing on-demand coaching, referrals, and reminders, Suzy may extend the reach of care teams, alleviate provider burden, and enhance patient engagement. Additional work is needed to understand how to best integrate Suzy into patients’ recovery journeys to ensure human support remains accessible and prioritized. LLM evaluation was based on expert testing and safety review. Clinical effectiveness, including the impact on substance use, was not evaluated. Next steps include evaluating the LLM chatbot in real-world settings with larger samples and assessing its efficacy in reducing substance use.
- Open Access
- 11 March 2024
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 8 (1) , e55205
Designing an App to Support Measurement-Based Peer Supervision of Frontline Health Workers Delivering Brief Psychosocial Interventions in Texas: Multimethod Study
Background: The unmet need for mental health care affects millions of Americans. A growing body of evidence in implementation science supports the effectiveness of task sharing in the delivery of brief psychosocial interventions. The digitization of training and processes supporting supervision can rapidly scale up task-shared interventions and enable frontline health workers (FLWs) to learn, master, and deliver interventions with quality and support. Objective: We aimed to assess the perceived feasibility and acceptability of a novel mobile and web app designed and adapted to support the supervision, training, and quality assurance of FLWs delivering brief psychosocial interventions. Methods: We followed human-centered design principles to adapt a prototype app for FLWs delivering brief psychosocial interventions for depression, drawing from an app previously designed for use in rural India. Using a multimethod approach, we conducted focus group sessions comprising usability testing and group interviews with FLWs recruited from a large health system in Texas to assess the feasibility and acceptability of the app. The positive System Usability Scale was used to determine the app’s overall usability. We also calculated the participants’ likelihood of recommending the app to others using ratings of 0 to 10 from least to most likely (net promoter score). Focus group transcripts were coded and analyzed thematically, and recommendations were summarized across 4 key domains. Results: A total of 18 FLWs varying in role and experience with client care participated in the study. Participants found the app to be usable, with an average System Usability Scale score of 72.5 (SD 18.1), consistent with the industry benchmark of 68. Participants’ likelihood of recommending the app ranged from 5 to 10, yielding a net promoter score of 0, indicating medium acceptability. Overall impressions of the app from participants were positive. Most participants (15/18, 83%) found the app easy to access and navigate. The app was considered important to support FLWs in delivering high-quality mental health care services. Participants felt that the app could provide more structure to FLW training and supervision processes through the systematic collection and facilitation of performance-related feedback. Key concerns included privacy-related and time constraints regarding implementing a separate peer supervision mechanism that may add to FLWs’ workloads. Conclusions: We designed, built, and tested a usable, functional mobile and web app prototype that supports FLW-delivered psychosocial interventions in the United States through a structured supervision mechanism and systematic collection and review of performance measures. The app has the potential to scale the work of FLWs tasked with delivering these interventions to the hardest-to-reach communities they serve. The results of this project will inform future work to evaluate the app’s use and efficacy in real-world settings to support task-shared mental health programs across the United States.
- 11 March 2024
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 8 (1) , e55205
Open AccessDesigning an App to Support Measurement-Based Peer Supervision of Frontline Health Workers Delivering Brief Psychosocial Interventions in Texas: Multimethod Study
Background: The unmet need for mental health care affects millions of Americans. A growing body of evidence in implementation science supports the effectiveness of task sharing in the delivery of brief psychosocial interventions. The digitization of training and processes supporting supervision can rapidly scale up task-shared interventions and enable frontline health workers (FLWs) to learn, master, and deliver interventions with quality and support. Objective: We aimed to assess the perceived feasibility and acceptability of a novel mobile and web app designed and adapted to support the supervision, training, and quality assurance of FLWs delivering brief psychosocial interventions. Methods: We followed human-centered design principles to adapt a prototype app for FLWs delivering brief psychosocial interventions for depression, drawing from an app previously designed for use in rural India. Using a multimethod approach, we conducted focus group sessions comprising usability testing and group interviews with FLWs recruited from a large health system in Texas to assess the feasibility and acceptability of the app. The positive System Usability Scale was used to determine the app’s overall usability. We also calculated the participants’ likelihood of recommending the app to others using ratings of 0 to 10 from least to most likely (net promoter score). Focus group transcripts were coded and analyzed thematically, and recommendations were summarized across 4 key domains. Results: A total of 18 FLWs varying in role and experience with client care participated in the study. Participants found the app to be usable, with an average System Usability Scale score of 72.5 (SD 18.1), consistent with the industry benchmark of 68. Participants’ likelihood of recommending the app ranged from 5 to 10, yielding a net promoter score of 0, indicating medium acceptability. Overall impressions of the app from participants were positive. Most participants (15/18, 83%) found the app easy to access and navigate. The app was considered important to support FLWs in delivering high-quality mental health care services. Participants felt that the app could provide more structure to FLW training and supervision processes through the systematic collection and facilitation of performance-related feedback. Key concerns included privacy-related and time constraints regarding implementing a separate peer supervision mechanism that may add to FLWs’ workloads. Conclusions: We designed, built, and tested a usable, functional mobile and web app prototype that supports FLW-delivered psychosocial interventions in the United States through a structured supervision mechanism and systematic collection and review of performance measures. The app has the potential to scale the work of FLWs tasked with delivering these interventions to the hardest-to-reach communities they serve. The results of this project will inform future work to evaluate the app’s use and efficacy in real-world settings to support task-shared mental health programs across the United States.
- Open Access
- 5 December 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Designing an App to Support Measurement-Based Peer Supervision of Frontline Health Workers Delivering Brief Psychosocial Interventions in Texas: Multimethod Study (Preprint)
BACKGROUND The unmet need for mental health care affects millions of Americans. A growing body of evidence in implementation science supports the effectiveness of task sharing in the delivery of brief psychosocial interventions. The digitization of training and processes supporting supervision can rapidly scale up task-shared interventions and enable frontline health workers (FLWs) to learn, master, and deliver interventions with quality and support. OBJECTIVE We aimed to assess the perceived feasibility and acceptability of a novel mobile and web app designed and adapted to support the supervision, training, and quality assurance of FLWs delivering brief psychosocial interventions. METHODS We followed human-centered design principles to adapt a prototype app for FLWs delivering brief psychosocial interventions for depression, drawing from an app previously designed for use in rural India. Using a multimethod approach, we conducted focus group sessions comprising usability testing and group interviews with FLWs recruited from a large health system in Texas to assess the feasibility and acceptability of the app. The positive System Usability Scale was used to determine the app’s overall usability. We also calculated the participants’ likelihood of recommending the app to others using ratings of 0 to 10 from least to most likely (net promoter score). Focus group transcripts were coded and analyzed thematically, and recommendations were summarized across 4 key domains. RESULTS A total of 18 FLWs varying in role and experience with client care participated in the study. Participants found the app to be usable, with an average System Usability Scale score of 72.5 (SD 18.1), consistent with the industry benchmark of 68. Participants’ likelihood of recommending the app ranged from 5 to 10, yielding a net promoter score of 0, indicating medium acceptability. Overall impressions of the app from participants were positive. Most participants (15/18, 83%) found the app easy to access and navigate. The app was considered important to support FLWs in delivering high-quality mental health care services. Participants felt that the app could provide more structure to FLW training and supervision processes through the systematic collection and facilitation of performance-related feedback. Key concerns included privacy-related and time constraints regarding implementing a separate peer supervision mechanism that may add to FLWs’ workloads. CONCLUSIONS We designed, built, and tested a usable, functional mobile and web app prototype that supports FLW-delivered psychosocial interventions in the United States through a structured supervision mechanism and systematic collection and review of performance measures. The app has the potential to scale the work of FLWs tasked with delivering these interventions to the hardest-to-reach communities they serve. The results of this project will inform future work to evaluate the app’s use and efficacy in real-world settings to support task-shared mental health programs across the United States.
- 5 December 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessDesigning an App to Support Measurement-Based Peer Supervision of Frontline Health Workers Delivering Brief Psychosocial Interventions in Texas: Multimethod Study (Preprint)
BACKGROUND The unmet need for mental health care affects millions of Americans. A growing body of evidence in implementation science supports the effectiveness of task sharing in the delivery of brief psychosocial interventions. The digitization of training and processes supporting supervision can rapidly scale up task-shared interventions and enable frontline health workers (FLWs) to learn, master, and deliver interventions with quality and support. OBJECTIVE We aimed to assess the perceived feasibility and acceptability of a novel mobile and web app designed and adapted to support the supervision, training, and quality assurance of FLWs delivering brief psychosocial interventions. METHODS We followed human-centered design principles to adapt a prototype app for FLWs delivering brief psychosocial interventions for depression, drawing from an app previously designed for use in rural India. Using a multimethod approach, we conducted focus group sessions comprising usability testing and group interviews with FLWs recruited from a large health system in Texas to assess the feasibility and acceptability of the app. The positive System Usability Scale was used to determine the app’s overall usability. We also calculated the participants’ likelihood of recommending the app to others using ratings of 0 to 10 from least to most likely (net promoter score). Focus group transcripts were coded and analyzed thematically, and recommendations were summarized across 4 key domains. RESULTS A total of 18 FLWs varying in role and experience with client care participated in the study. Participants found the app to be usable, with an average System Usability Scale score of 72.5 (SD 18.1), consistent with the industry benchmark of 68. Participants’ likelihood of recommending the app ranged from 5 to 10, yielding a net promoter score of 0, indicating medium acceptability. Overall impressions of the app from participants were positive. Most participants (15/18, 83%) found the app easy to access and navigate. The app was considered important to support FLWs in delivering high-quality mental health care services. Participants felt that the app could provide more structure to FLW training and supervision processes through the systematic collection and facilitation of performance-related feedback. Key concerns included privacy-related and time constraints regarding implementing a separate peer supervision mechanism that may add to FLWs’ workloads. CONCLUSIONS We designed, built, and tested a usable, functional mobile and web app prototype that supports FLW-delivered psychosocial interventions in the United States through a structured supervision mechanism and systematic collection and review of performance measures. The app has the potential to scale the work of FLWs tasked with delivering these interventions to the hardest-to-reach communities they serve. The results of this project will inform future work to evaluate the app’s use and efficacy in real-world settings to support task-shared mental health programs across the United States.
- 1 September 2023
- Published by American Society of Clinical Oncology (ASCO) in JCO Clinical Cancer Informatics
- Vol. 7 (7) , e2300022
Designing a Digital Tool to Inform Oncology Practices About Cancer Treatment Cost Burdens
Paper describes our development of a web-based tool to make Medicare patient prices for cancer drugs known to their care teams.
- 1 September 2023
- Published by American Society of Clinical Oncology (ASCO) in JCO Clinical Cancer Informatics
- Vol. 7 (7) , e2300022
Designing a Digital Tool to Inform Oncology Practices About Cancer Treatment Cost Burdens
Paper describes our development of a web-based tool to make Medicare patient prices for cancer drugs known to their care teams.