Grant Graziani
Research Subjects
Research Interests
Grant Graziani has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Sarah Kunkle
Headspace, Santa Monica, CA, United States
9 shared publications - Emily Shih
Ginger San Francisco, CA United States
8 shared publications - Brandon S. Aylward
Ginger San Francisco, CA United States
6 shared publications - Akhil Chaturvedi
Headspace Health, San Francisco, CA, United States
1 shared publication - Bobby Manuel
Headspace Health, San Francisco, CA, United States
1 shared publication Brandon Aylward
1 shared publication
Latest Publications
- Open Access
- 19 January 2023
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 7, e38831
Content Recommendation Systems in Web-Based Mental Health Care: Real-world Application and Formative Evaluation
Background: Recommender systems have great potential in mental health care to personalize self-guided content for patients, allowing them to supplement their mental health treatment in a scalable way. Objective: In this paper, we describe and evaluate 2 knowledge-based content recommendation systems as parts of Ginger, an on-demand mental health platform, to bolster engagement in self-guided mental health content. Methods: We developed two algorithms to provide content recommendations in the Ginger mental health smartphone app: (1) one that uses users' responses to app onboarding questions to recommend content cards and (2) one that uses the semantic similarity between the transcript of a coaching conversation and the description of content cards to make recommendations after every session. As a measure of success for these recommendation algorithms, we examined the relevance of content cards to users’ conversations with their coach and completion rates of selected content within the app measured over 14,018 users. Results: In a real-world setting, content consumed in the recommendations section (or “Explore” in the app) had the highest completion rates (3353/7871, 42.6%) compared to other sections of the app, which had an average completion rate of 37.35% (21,982/58,614; P<.001). Within the app’s recommendations section, conversation-based content recommendations had 11.4% (1108/2364) higher completion rates per card than onboarding response-based recommendations (1712/4067; P=.003) and 26.1% higher than random recommendations (534/1440; P=.005). Studied via subject matter experts’ annotations, conversation-based recommendations had a 16.1% higher relevance rate for the top 5 recommended cards, averaged across sessions of varying lengths, compared to a random control (110 conversational sessions). Finally, it was observed that both age and gender variables were sensitive to different recommendation methods, with responsiveness to personalized recommendations being higher if the users were older than 35 years or identified as male. Conclusions: Recommender systems can help scale and supplement digital mental health care with personalized content and self-care recommendations. Onboarding-based recommendations are ideal for “cold starting” the process of recommending content for new users and users that tend to use the app just for content but not for therapy or coaching. The conversation-based recommendation algorithm allows for dynamic recommendations based on information gathered during coaching sessions, which is a critical capability, given the changing nature of mental health needs during treatment. The proposed algorithms are just one step toward the direction of outcome-driven personalization in mental health. Our future work will involve a robust causal evaluation of these algorithms using randomized controlled trials, along with consumer feedback–driven improvement of these algorithms, to drive better clinical outcomes.
- 19 January 2023
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 7, e38831
Open AccessContent Recommendation Systems in Web-Based Mental Health Care: Real-world Application and Formative Evaluation
Background: Recommender systems have great potential in mental health care to personalize self-guided content for patients, allowing them to supplement their mental health treatment in a scalable way. Objective: In this paper, we describe and evaluate 2 knowledge-based content recommendation systems as parts of Ginger, an on-demand mental health platform, to bolster engagement in self-guided mental health content. Methods: We developed two algorithms to provide content recommendations in the Ginger mental health smartphone app: (1) one that uses users' responses to app onboarding questions to recommend content cards and (2) one that uses the semantic similarity between the transcript of a coaching conversation and the description of content cards to make recommendations after every session. As a measure of success for these recommendation algorithms, we examined the relevance of content cards to users’ conversations with their coach and completion rates of selected content within the app measured over 14,018 users. Results: In a real-world setting, content consumed in the recommendations section (or “Explore” in the app) had the highest completion rates (3353/7871, 42.6%) compared to other sections of the app, which had an average completion rate of 37.35% (21,982/58,614; P<.001). Within the app’s recommendations section, conversation-based content recommendations had 11.4% (1108/2364) higher completion rates per card than onboarding response-based recommendations (1712/4067; P=.003) and 26.1% higher than random recommendations (534/1440; P=.005). Studied via subject matter experts’ annotations, conversation-based recommendations had a 16.1% higher relevance rate for the top 5 recommended cards, averaged across sessions of varying lengths, compared to a random control (110 conversational sessions). Finally, it was observed that both age and gender variables were sensitive to different recommendation methods, with responsiveness to personalized recommendations being higher if the users were older than 35 years or identified as male. Conclusions: Recommender systems can help scale and supplement digital mental health care with personalized content and self-care recommendations. Onboarding-based recommendations are ideal for “cold starting” the process of recommending content for new users and users that tend to use the app just for content but not for therapy or coaching. The conversation-based recommendation algorithm allows for dynamic recommendations based on information gathered during coaching sessions, which is a critical capability, given the changing nature of mental health needs during treatment. The proposed algorithms are just one step toward the direction of outcome-driven personalization in mental health. Our future work will involve a robust causal evaluation of these algorithms using randomized controlled trials, along with consumer feedback–driven improvement of these algorithms, to drive better clinical outcomes.
- Open Access
- 28 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e37169
Changes in Resilience Following Engagement With a Virtual Mental Health System: Real-world Observational Study
Background: Digital health services can serve as scalable solutions to address the growing demand for mental health care. However, more research is needed to better understand the association between engagement with care and improvements in subclinical outcomes. Objective: This study aims to fill this research gap by examining the relationship between members' engagement with the Ginger platform and changes in their psychological resilience. Methods: We conducted a retrospective observational study of 3272 members who accessed Ginger, an on-demand mental health service, between January 2021 and November 2021. Each member completed the 10-item Connor-Davidson Resilience Scale questionnaire, a measure of psychological resilience, at baseline and again during a 6- to 16-week follow-up window. Depression and anxiety symptoms (9-item Patient Health Questionnaire and 7-item Generalized Anxiety Disorder) were also measured. Linear regression was used to identify the association between engagement with Ginger's multiple care modalities and changes in resilience. Moderator analysis was conducted to test whether clinical depression or anxiety at baseline moderated the relationship between engagement level and changes in resilience. Results: Of the 3272 members, 2683 (82%) reported low resilience at baseline. The mean change in resilience was 0.77 (SD 5.50) points. Linear regression models showed that age and census region did not predict changes in resilience; however, male members showed larger improvements (coefficient=0.58; P=.04). Baseline mental health outcomes, including resilience and depression and anxiety symptoms, were strong predictors of changes in resilience. Every point decrease in baseline resilience is associated with a 0.28-point increase in change in resilience (P<.001), and members with no or mild depression and anxiety at baseline saw changes in resilience that were 1.44 points (P<.001) larger than their clinical counterparts. Engagement with the Ginger system predicted changes in resilience. Members who engaged with Ginger coaching, clinical services, or both improved their resilience by 1.82, 1.55, and 1.40 points, respectively (P<.001), more than those who only engaged with Ginger content. Screening negative for moderate to severe depression and anxiety at baseline was associated with larger improvements in resilience (coefficient=1.30; P<.001); however, subclinical status was not shown to be a moderator for the association between level of engagement and changes in resilience. Conclusions: Engagement with Ginger services was associated with improvements in resilience. Members who engaged in coaching or clinical care had significantly larger improvements compared with those who only engaged in self-guided content, regardless of whether a member screened positive for clinical depression or anxiety at baseline.
- 28 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e37169
Open AccessChanges in Resilience Following Engagement With a Virtual Mental Health System: Real-world Observational Study
Background: Digital health services can serve as scalable solutions to address the growing demand for mental health care. However, more research is needed to better understand the association between engagement with care and improvements in subclinical outcomes. Objective: This study aims to fill this research gap by examining the relationship between members' engagement with the Ginger platform and changes in their psychological resilience. Methods: We conducted a retrospective observational study of 3272 members who accessed Ginger, an on-demand mental health service, between January 2021 and November 2021. Each member completed the 10-item Connor-Davidson Resilience Scale questionnaire, a measure of psychological resilience, at baseline and again during a 6- to 16-week follow-up window. Depression and anxiety symptoms (9-item Patient Health Questionnaire and 7-item Generalized Anxiety Disorder) were also measured. Linear regression was used to identify the association between engagement with Ginger's multiple care modalities and changes in resilience. Moderator analysis was conducted to test whether clinical depression or anxiety at baseline moderated the relationship between engagement level and changes in resilience. Results: Of the 3272 members, 2683 (82%) reported low resilience at baseline. The mean change in resilience was 0.77 (SD 5.50) points. Linear regression models showed that age and census region did not predict changes in resilience; however, male members showed larger improvements (coefficient=0.58; P=.04). Baseline mental health outcomes, including resilience and depression and anxiety symptoms, were strong predictors of changes in resilience. Every point decrease in baseline resilience is associated with a 0.28-point increase in change in resilience (P<.001), and members with no or mild depression and anxiety at baseline saw changes in resilience that were 1.44 points (P<.001) larger than their clinical counterparts. Engagement with the Ginger system predicted changes in resilience. Members who engaged with Ginger coaching, clinical services, or both improved their resilience by 1.82, 1.55, and 1.40 points, respectively (P<.001), more than those who only engaged with Ginger content. Screening negative for moderate to severe depression and anxiety at baseline was associated with larger improvements in resilience (coefficient=1.30; P<.001); however, subclinical status was not shown to be a moderator for the association between level of engagement and changes in resilience. Conclusions: Engagement with Ginger services was associated with improvements in resilience. Members who engaged in coaching or clinical care had significantly larger improvements compared with those who only engaged in self-guided content, regardless of whether a member screened positive for clinical depression or anxiety at baseline.
- Open Access
- 22 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e36956
Association Between Care Modality and Use With Treatment Response Among Members Accessing Virtual Mental Health Services: Real-world Observational Study
Background: There is a growing bottleneck in mental health care, as the demand for services has outpaced the availability of mental health professionals. Consequently, many health systems have shifted to teletherapy as a scalable approach to increasing accessibility to care. Within these care models, various treatment modalities (eg, coaching and clinical care) are used to deliver support for anxiety and depression. However, more research is needed to better understand the differences in treatment responses. Objective: The purpose of this study was to examine the association between different care modalities and the levels of use with symptom score changes for members seeking virtual care services. Methods: We conducted an observational study of 4219 members who accessed Ginger, an on-demand mental health service, between September 2020 and September 2021. Using a mobile app, members can access text-based behavioral health coaching and virtual clinical services. This study focused on members with clinically elevated depression or anxiety levels at baseline. Logistic regressions were used to assess the association between care modalities and the levels of use with treatment response in depression and anxiety, using the Patient Health Questionnaire and Generalized Anxiety Disorder Assessment, respectively. Results: Of the 4219 members, 1623 (38.47%) demonstrated a full response to depression, and 1684 (39.91%) demonstrated a full response to anxiety. Members who completed care (ie, text-based coaching, virtual clinical therapy, hybrid of coaching, and clinical care) beyond the introductory session showed significantly increased odds of a full response compared with those who completed only limited care. Members who completed a hybrid of care had the highest odds of improvement; the odds of showing a full response in depression were 2.31 times higher (95% CI 1.91-2.80; P<.001) and in anxiety were 2.23 times higher (95% CI 1.84-2.70; P<.001) compared with members who completed limited care. For members who completed only coaching or clinical care, the largest effects were observed among those with high use. For members who completed a hybrid care program, we observed similar treatment responses across all levels of use. Conclusions: Our real-world study found that members who completed text-based coaching achieved full treatment responses at similar rates compared with members who completed virtual clinical care and members who completed a hybrid of care. There were no significant differences in the predicted probabilities of full treatment response between coaching and clinical care. Generally, the odds for a full response were highest among members with high use within each care modality; however, there were no differences in full-response treatment odds across levels of use with hybrid care. The results support the utility of digital behavioral health interventions and further highlight text-based coaching protocols as an accessible and suitable option when considering virtual care for treating anxiety and depression.
- 22 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e36956
Open AccessAssociation Between Care Modality and Use With Treatment Response Among Members Accessing Virtual Mental Health Services: Real-world Observational Study
Background: There is a growing bottleneck in mental health care, as the demand for services has outpaced the availability of mental health professionals. Consequently, many health systems have shifted to teletherapy as a scalable approach to increasing accessibility to care. Within these care models, various treatment modalities (eg, coaching and clinical care) are used to deliver support for anxiety and depression. However, more research is needed to better understand the differences in treatment responses. Objective: The purpose of this study was to examine the association between different care modalities and the levels of use with symptom score changes for members seeking virtual care services. Methods: We conducted an observational study of 4219 members who accessed Ginger, an on-demand mental health service, between September 2020 and September 2021. Using a mobile app, members can access text-based behavioral health coaching and virtual clinical services. This study focused on members with clinically elevated depression or anxiety levels at baseline. Logistic regressions were used to assess the association between care modalities and the levels of use with treatment response in depression and anxiety, using the Patient Health Questionnaire and Generalized Anxiety Disorder Assessment, respectively. Results: Of the 4219 members, 1623 (38.47%) demonstrated a full response to depression, and 1684 (39.91%) demonstrated a full response to anxiety. Members who completed care (ie, text-based coaching, virtual clinical therapy, hybrid of coaching, and clinical care) beyond the introductory session showed significantly increased odds of a full response compared with those who completed only limited care. Members who completed a hybrid of care had the highest odds of improvement; the odds of showing a full response in depression were 2.31 times higher (95% CI 1.91-2.80; P<.001) and in anxiety were 2.23 times higher (95% CI 1.84-2.70; P<.001) compared with members who completed limited care. For members who completed only coaching or clinical care, the largest effects were observed among those with high use. For members who completed a hybrid care program, we observed similar treatment responses across all levels of use. Conclusions: Our real-world study found that members who completed text-based coaching achieved full treatment responses at similar rates compared with members who completed virtual clinical care and members who completed a hybrid of care. There were no significant differences in the predicted probabilities of full treatment response between coaching and clinical care. Generally, the odds for a full response were highest among members with high use within each care modality; however, there were no differences in full-response treatment odds across levels of use with hybrid care. The results support the utility of digital behavioral health interventions and further highlight text-based coaching protocols as an accessible and suitable option when considering virtual care for treating anxiety and depression.
- Open Access
- 8 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e35352
Health-Related Quality of Life Among Members Using an On Demand Behavioral Health Platform: Pilot Observational Study
Background: Despite the well-known adverse health conditions and negative economic outcomes associated with mental health problems, accessing treatment is difficult due to reasons such as availability and cost. As a solution, digital mental health services have flooded the industry, and new studies are quickly emerging that support their potential as an accessible and cost-effective way to improve mental health outcomes. However, many mental health platforms typically use clinical tools such as the Patient Health Questionnaire-9 (PHQ-9) or General Anxiety Disorder-7 (GAD-7). Yet, many individuals that seek out care do not have clinical symptomatology and thus, traditional clinical measures may not adequately capture symptom improvement in general well-being. As an alternative, this study used the health-related quality of life (HRQoL) tool from the Centers for Disease Control and Prevention “Healthy Days” measure. This subjective measure of well-being is an effective way to capture HRQoL and might be better suited as an outcome measure for treatments that include both clinical and subclinical individuals. Objective: The purpose of this study was to describe changes in HRQoL in clinical and subclinical members assessing virtual care and to examine the association between text-based behavioral coaching and virtual clinical sessions with changes in HRQoL. Methods: A total of 288 members completed the 4-item HRQoL measure at baseline and at 1 month following use of the Ginger on demand behavioral health platform. Baseline anxiety and depression levels were collected using the GAD-7 and PHQ-9, respectively. Results: Members completed on average 1.92 (SD 2.16) coaching sessions and 0.91 (SD 1.37) clinical sessions during the assessment month. Paired samples t tests revealed significant reductions in the average number of unhealthy mental health days between baseline (mean 16, SD 8.77 days) and follow-up (mean 13.2, SD 9.02 days; t287=5.73; P<.001), and in the average number of days adversely impacted (meanbaseline 10.9, meanfollow-up 8.19; t287=6.26; P<.001). Both subclinical members (t103=3.04; P=.003) and clinical members (t183=5.5; P<.001) demonstrated significant improvements through reductions in adversely impacted days over a month. Clinical members also demonstrated significant improvements through reductions in unhealthy mental health days (t183=5.82; P<.001). Finally, member engagement with virtual clinical sessions significantly predicted changes in unhealthy mental health days (B=–0.96; P=.04). Conclusions: To our knowledge, this study is one of the first to use the HRQoL measure as an outcome in an evaluation of a digital behavioral health platform. Using real-world longitudinal data, our preliminary yet promising results show that short-term engagement with virtual care can be an effective means to improve HRQoL for members with subclinical and clinical symptoms. Further follow-up of reported HRQoL over several months is needed.
- 8 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e35352
Open AccessHealth-Related Quality of Life Among Members Using an On Demand Behavioral Health Platform: Pilot Observational Study
Background: Despite the well-known adverse health conditions and negative economic outcomes associated with mental health problems, accessing treatment is difficult due to reasons such as availability and cost. As a solution, digital mental health services have flooded the industry, and new studies are quickly emerging that support their potential as an accessible and cost-effective way to improve mental health outcomes. However, many mental health platforms typically use clinical tools such as the Patient Health Questionnaire-9 (PHQ-9) or General Anxiety Disorder-7 (GAD-7). Yet, many individuals that seek out care do not have clinical symptomatology and thus, traditional clinical measures may not adequately capture symptom improvement in general well-being. As an alternative, this study used the health-related quality of life (HRQoL) tool from the Centers for Disease Control and Prevention “Healthy Days” measure. This subjective measure of well-being is an effective way to capture HRQoL and might be better suited as an outcome measure for treatments that include both clinical and subclinical individuals. Objective: The purpose of this study was to describe changes in HRQoL in clinical and subclinical members assessing virtual care and to examine the association between text-based behavioral coaching and virtual clinical sessions with changes in HRQoL. Methods: A total of 288 members completed the 4-item HRQoL measure at baseline and at 1 month following use of the Ginger on demand behavioral health platform. Baseline anxiety and depression levels were collected using the GAD-7 and PHQ-9, respectively. Results: Members completed on average 1.92 (SD 2.16) coaching sessions and 0.91 (SD 1.37) clinical sessions during the assessment month. Paired samples t tests revealed significant reductions in the average number of unhealthy mental health days between baseline (mean 16, SD 8.77 days) and follow-up (mean 13.2, SD 9.02 days; t287=5.73; P<.001), and in the average number of days adversely impacted (meanbaseline 10.9, meanfollow-up 8.19; t287=6.26; P<.001). Both subclinical members (t103=3.04; P=.003) and clinical members (t183=5.5; P<.001) demonstrated significant improvements through reductions in adversely impacted days over a month. Clinical members also demonstrated significant improvements through reductions in unhealthy mental health days (t183=5.82; P<.001). Finally, member engagement with virtual clinical sessions significantly predicted changes in unhealthy mental health days (B=–0.96; P=.04). Conclusions: To our knowledge, this study is one of the first to use the HRQoL measure as an outcome in an evaluation of a digital behavioral health platform. Using real-world longitudinal data, our preliminary yet promising results show that short-term engagement with virtual care can be an effective means to improve HRQoL for members with subclinical and clinical symptoms. Further follow-up of reported HRQoL over several months is needed.
- Open Access
- 31 March 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (3) , e34283
Resilience in 2021—Descriptive Analysis of Individuals Accessing Virtual Mental Health Services: Retrospective Observational Study
Background: Psychological resilience has been extensively studied by developmental researchers, and there is a growing body of literature regarding its role in psychiatry and psychopathology research and practice. This study contributes to this growing literature by providing real-world evidence on the relationship between resilience and clinical symptoms among a large sample of employed Americans. Objective: This study aimed to describe resilience levels in individuals accessing Ginger, a virtual mental health system, in addition to the association of resilience with demographic characteristics, baseline depression, and anxiety symptoms. Methods: We conducted a retrospective observational study of 9165 members who signed up for Ginger and completed a baseline survey between January 1 and August 5, 2021. We used multivariate regression models to test for associations between baseline resilience and other member characteristics. Results: Baseline resilience scores centered on a mean of 23.84 (SD 6.56) and median of 24 (IQR 8) out of 40, with 81.0% (7424/9165) of the sample having low resilience at baseline. Despite having relatively higher resilience scores, members with no or mild depression or anxiety still had low resilience scores on average. Self-reported suicidal ideation was associated with lower resilience. Conclusions: Overall, members had low baseline resilience, similar to resilience levels observed in trauma survivors in prior studies. Younger members and those with higher levels of depression and anxiety at intake reported lower levels of resilience at baseline. Notably, members with no or mild depression or anxiety still had low resilience scores on average, suggesting a need for mental health support among individuals who might not typically be recommended for treatment based on traditional clinical assessments, such as the 9-item Patient Health Questionnaire (PHQ-9) and the 7-item Generalized Anxiety Disorder scale (GAD-7). Two suggestions for topics of future research are to develop treatment recommendations based on the Connor-Davidson Resilience Scale and to understand the interaction between resilience levels and symptom-based outcome measures, such as the PHQ-9 and the GAD-7.
- 31 March 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (3) , e34283
Open AccessResilience in 2021—Descriptive Analysis of Individuals Accessing Virtual Mental Health Services: Retrospective Observational Study
Background: Psychological resilience has been extensively studied by developmental researchers, and there is a growing body of literature regarding its role in psychiatry and psychopathology research and practice. This study contributes to this growing literature by providing real-world evidence on the relationship between resilience and clinical symptoms among a large sample of employed Americans. Objective: This study aimed to describe resilience levels in individuals accessing Ginger, a virtual mental health system, in addition to the association of resilience with demographic characteristics, baseline depression, and anxiety symptoms. Methods: We conducted a retrospective observational study of 9165 members who signed up for Ginger and completed a baseline survey between January 1 and August 5, 2021. We used multivariate regression models to test for associations between baseline resilience and other member characteristics. Results: Baseline resilience scores centered on a mean of 23.84 (SD 6.56) and median of 24 (IQR 8) out of 40, with 81.0% (7424/9165) of the sample having low resilience at baseline. Despite having relatively higher resilience scores, members with no or mild depression or anxiety still had low resilience scores on average. Self-reported suicidal ideation was associated with lower resilience. Conclusions: Overall, members had low baseline resilience, similar to resilience levels observed in trauma survivors in prior studies. Younger members and those with higher levels of depression and anxiety at intake reported lower levels of resilience at baseline. Notably, members with no or mild depression or anxiety still had low resilience scores on average, suggesting a need for mental health support among individuals who might not typically be recommended for treatment based on traditional clinical assessments, such as the 9-item Patient Health Questionnaire (PHQ-9) and the 7-item Generalized Anxiety Disorder scale (GAD-7). Two suggestions for topics of future research are to develop treatment recommendations based on the Connor-Davidson Resilience Scale and to understand the interaction between resilience levels and symptom-based outcome measures, such as the PHQ-9 and the GAD-7.
- Open Access
- 9 February 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Changes in Resilience Following Engagement with A Virtual Mental Health System: A Real World Observational Study (Preprint)
BACKGROUND Digital health services can serve as scalable solutions to address the growing demand of mental health care, yet more research is needed to better understand the association between engagement with care and improvements in subclinical outcomes. OBJECTIVE This study aims to fill this gap by studying the relationship between members’ engagement with the Ginger platform and changes in their psychological resilience. METHODS We conducted a retrospective observational study of 3,272 members who accessed Ginger, an on-demand mental health service, between January 2021 and November 2021. Each member completed the CD-RISC-10 questionnaire, a measure of psychological resilience, at baseline and then again during a 6 to 16 week follow-up window. Depression and anxiety symptoms (PHQ-9 and GAD-7) were also measured. Linear regressions were used to identify the association between engagement in Ginger’s multiple care modalities and changes in resilience. A moderator analysis was conducted to test whether clinical depression or anxiety at baseline is a moderator between engagement level and changes in resilience. RESULTS Of the 3,272 members, 82% reported low resilience at baseline. The mean change in resilience was 0.77 points (sd=5.50). Linear regression models show that age and census region do not predict changes in resilience; however, male members see larger improvements (coeff = 0.58, p < .05). Baseline mental health outcomes, including resilience and depression and anxiety symptoms, are stronger predictors of changes in resilience. Every point decrease in baseline resilience is associated with a 0.28 point increase in change in resilience (p < .01), and members with no or mild depression and anxiety at baseline saw a changes in resilience that were 1.44 points (p < .01) larger than their clinical counterparts. Engagement with the Ginger system predicted changes in resilience. Members who engaged with Ginger coaching, clinical services or both improved their resilience by 1.82, 1.55 and 1.40 points respectively (ps < .01) more than those who only engaged with Ginger content. Screening negative for moderate to severe depression and anxiety at baseline was associated with larger improvements in resilience (coeff= 1.30, p < .01); however, subclinical status was not shown to be a moderator for the association between level of engagement and changes in resilience. CONCLUSIONS Engagement with Ginger services was associated with improvements in resilience. Members who engaged with coaching or clinical care had significantly larger improvements compared to those who only engaged with self-guided content, regardless of whether a member screened positive for clinical depression or anxiety at baseline.
- 9 February 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessChanges in Resilience Following Engagement with A Virtual Mental Health System: A Real World Observational Study (Preprint)
BACKGROUND Digital health services can serve as scalable solutions to address the growing demand of mental health care, yet more research is needed to better understand the association between engagement with care and improvements in subclinical outcomes. OBJECTIVE This study aims to fill this gap by studying the relationship between members’ engagement with the Ginger platform and changes in their psychological resilience. METHODS We conducted a retrospective observational study of 3,272 members who accessed Ginger, an on-demand mental health service, between January 2021 and November 2021. Each member completed the CD-RISC-10 questionnaire, a measure of psychological resilience, at baseline and then again during a 6 to 16 week follow-up window. Depression and anxiety symptoms (PHQ-9 and GAD-7) were also measured. Linear regressions were used to identify the association between engagement in Ginger’s multiple care modalities and changes in resilience. A moderator analysis was conducted to test whether clinical depression or anxiety at baseline is a moderator between engagement level and changes in resilience. RESULTS Of the 3,272 members, 82% reported low resilience at baseline. The mean change in resilience was 0.77 points (sd=5.50). Linear regression models show that age and census region do not predict changes in resilience; however, male members see larger improvements (coeff = 0.58, p < .05). Baseline mental health outcomes, including resilience and depression and anxiety symptoms, are stronger predictors of changes in resilience. Every point decrease in baseline resilience is associated with a 0.28 point increase in change in resilience (p < .01), and members with no or mild depression and anxiety at baseline saw a changes in resilience that were 1.44 points (p < .01) larger than their clinical counterparts. Engagement with the Ginger system predicted changes in resilience. Members who engaged with Ginger coaching, clinical services or both improved their resilience by 1.82, 1.55 and 1.40 points respectively (ps < .01) more than those who only engaged with Ginger content. Screening negative for moderate to severe depression and anxiety at baseline was associated with larger improvements in resilience (coeff= 1.30, p < .01); however, subclinical status was not shown to be a moderator for the association between level of engagement and changes in resilience. CONCLUSIONS Engagement with Ginger services was associated with improvements in resilience. Members who engaged with coaching or clinical care had significantly larger improvements compared to those who only engaged with self-guided content, regardless of whether a member screened positive for clinical depression or anxiety at baseline.
- Open Access
- 1 February 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Association Between Care Modality and Use With Treatment Response Among Members Accessing Virtual Mental Health Services: Real-world Observational Study (Preprint)
BACKGROUND There is a growing bottleneck in mental health care, as the demand for services has outpaced the availability of mental health professionals. Consequently, many health systems have shifted to teletherapy as a scalable approach to increasing accessibility to care. Within these care models, various treatment modalities (eg, coaching and clinical care) are used to deliver support for anxiety and depression. However, more research is needed to better understand the differences in treatment responses. OBJECTIVE The purpose of this study was to examine the association between different care modalities and the levels of use with symptom score changes for members seeking virtual care services. METHODS We conducted an observational study of 4219 members who accessed Ginger, an on-demand mental health service, between September 2020 and September 2021. Using a mobile app, members can access text-based behavioral health coaching and virtual clinical services. This study focused on members with clinically elevated depression or anxiety levels at baseline. Logistic regressions were used to assess the association between care modalities and the levels of use with treatment response in depression and anxiety, using the Patient Health Questionnaire and Generalized Anxiety Disorder Assessment, respectively. RESULTS Of the 4219 members, 1623 (38.47%) demonstrated a full response to depression, and 1684 (39.91%) demonstrated a full response to anxiety. Members who completed care (ie, text-based coaching, virtual clinical therapy, hybrid of coaching, and clinical care) beyond the introductory session showed significantly increased odds of a full response compared with those who completed only limited care. Members who completed a hybrid of care had the highest odds of improvement; the odds of showing a full response in depression were 2.31 times higher (95% CI 1.91-2.80; P<.001) and in anxiety were 2.23 times higher (95% CI 1.84-2.70; P<.001) compared with members who completed limited care. For members who completed only coaching or clinical care, the largest effects were observed among those with high use. For members who completed a hybrid care program, we observed similar treatment responses across all levels of use. CONCLUSIONS Our real-world study found that members who completed text-based coaching achieved full treatment responses at similar rates compared with members who completed virtual clinical care and members who completed a hybrid of care. There were no significant differences in the predicted probabilities of full treatment response between coaching and clinical care. Generally, the odds for a full response were highest among members with high use within each care modality; however, there were no differences in full-response treatment odds across levels of use with hybrid care. The results support the utility of digital behavioral health interventions and further highlight text-based coaching protocols as an accessible and suitable option when considering virtual care for treating anxiety and depression.
- 1 February 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessAssociation Between Care Modality and Use With Treatment Response Among Members Accessing Virtual Mental Health Services: Real-world Observational Study (Preprint)
BACKGROUND There is a growing bottleneck in mental health care, as the demand for services has outpaced the availability of mental health professionals. Consequently, many health systems have shifted to teletherapy as a scalable approach to increasing accessibility to care. Within these care models, various treatment modalities (eg, coaching and clinical care) are used to deliver support for anxiety and depression. However, more research is needed to better understand the differences in treatment responses. OBJECTIVE The purpose of this study was to examine the association between different care modalities and the levels of use with symptom score changes for members seeking virtual care services. METHODS We conducted an observational study of 4219 members who accessed Ginger, an on-demand mental health service, between September 2020 and September 2021. Using a mobile app, members can access text-based behavioral health coaching and virtual clinical services. This study focused on members with clinically elevated depression or anxiety levels at baseline. Logistic regressions were used to assess the association between care modalities and the levels of use with treatment response in depression and anxiety, using the Patient Health Questionnaire and Generalized Anxiety Disorder Assessment, respectively. RESULTS Of the 4219 members, 1623 (38.47%) demonstrated a full response to depression, and 1684 (39.91%) demonstrated a full response to anxiety. Members who completed care (ie, text-based coaching, virtual clinical therapy, hybrid of coaching, and clinical care) beyond the introductory session showed significantly increased odds of a full response compared with those who completed only limited care. Members who completed a hybrid of care had the highest odds of improvement; the odds of showing a full response in depression were 2.31 times higher (95% CI 1.91-2.80; P<.001) and in anxiety were 2.23 times higher (95% CI 1.84-2.70; P<.001) compared with members who completed limited care. For members who completed only coaching or clinical care, the largest effects were observed among those with high use. For members who completed a hybrid care program, we observed similar treatment responses across all levels of use. CONCLUSIONS Our real-world study found that members who completed text-based coaching achieved full treatment responses at similar rates compared with members who completed virtual clinical care and members who completed a hybrid of care. There were no significant differences in the predicted probabilities of full treatment response between coaching and clinical care. Generally, the odds for a full response were highest among members with high use within each care modality; however, there were no differences in full-response treatment odds across levels of use with hybrid care. The results support the utility of digital behavioral health interventions and further highlight text-based coaching protocols as an accessible and suitable option when considering virtual care for treating anxiety and depression.
- Open Access
- 1 December 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Health-Related Quality of Life Among Members Using an On Demand Behavioral Health Platform: Pilot Observational Study (Preprint)
BACKGROUND Despite the well-known adverse health conditions and negative economic outcomes associated with mental health problems, accessing treatment is difficult due to reasons such as availability and cost. As a solution, digital mental health services have flooded the industry, and new studies are quickly emerging that support their potential as an accessible and cost-effective way to improve mental health outcomes. However, many mental health platforms typically use clinical tools such as the Patient Health Questionnaire-9 (PHQ-9) or General Anxiety Disorder-7 (GAD-7). Yet, many individuals that seek out care do not have clinical symptomatology and thus, traditional clinical measures may not adequately capture symptom improvement in general well-being. As an alternative, this study used the health-related quality of life (HRQoL) tool from the Centers for Disease Control and Prevention “Healthy Days” measure. This subjective measure of well-being is an effective way to capture HRQoL and might be better suited as an outcome measure for treatments that include both clinical and subclinical individuals. OBJECTIVE The purpose of this study was to describe changes in HRQoL in clinical and subclinical members assessing virtual care and to examine the association between text-based behavioral coaching and virtual clinical sessions with changes in HRQoL. METHODS A total of 288 members completed the 4-item HRQoL measure at baseline and at 1 month following use of the Ginger on demand behavioral health platform. Baseline anxiety and depression levels were collected using the GAD-7 and PHQ-9, respectively. RESULTS Members completed on average 1.92 (SD 2.16) coaching sessions and 0.91 (SD 1.37) clinical sessions during the assessment month. Paired samples t tests revealed significant reductions in the average number of unhealthy mental health days between baseline (mean 16, SD 8.77 days) and follow-up (mean 13.2, SD 9.02 days; t287=5.73; P<.001), and in the average number of days adversely impacted (meanbaseline 10.9, meanfollow-up 8.19; t287=6.26; P<.001). Both subclinical members (t103=3.04; P=.003) and clinical members (t183=5.5; P<.001) demonstrated significant improvements through reductions in adversely impacted days over a month. Clinical members also demonstrated significant improvements through reductions in unhealthy mental health days (t183=5.82; P<.001). Finally, member engagement with virtual clinical sessions significantly predicted changes in unhealthy mental health days (B=–0.96; P=.04). CONCLUSIONS To our knowledge, this study is one of the first to use the HRQoL measure as an outcome in an evaluation of a digital behavioral health platform. Using real-world longitudinal data, our preliminary yet promising results show that short-term engagement with virtual care can be an effective means to improve HRQoL for members with subclinical and clinical symptoms. Further follow-up of reported HRQoL over several months is needed.
- 1 December 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessHealth-Related Quality of Life Among Members Using an On Demand Behavioral Health Platform: Pilot Observational Study (Preprint)
BACKGROUND Despite the well-known adverse health conditions and negative economic outcomes associated with mental health problems, accessing treatment is difficult due to reasons such as availability and cost. As a solution, digital mental health services have flooded the industry, and new studies are quickly emerging that support their potential as an accessible and cost-effective way to improve mental health outcomes. However, many mental health platforms typically use clinical tools such as the Patient Health Questionnaire-9 (PHQ-9) or General Anxiety Disorder-7 (GAD-7). Yet, many individuals that seek out care do not have clinical symptomatology and thus, traditional clinical measures may not adequately capture symptom improvement in general well-being. As an alternative, this study used the health-related quality of life (HRQoL) tool from the Centers for Disease Control and Prevention “Healthy Days” measure. This subjective measure of well-being is an effective way to capture HRQoL and might be better suited as an outcome measure for treatments that include both clinical and subclinical individuals. OBJECTIVE The purpose of this study was to describe changes in HRQoL in clinical and subclinical members assessing virtual care and to examine the association between text-based behavioral coaching and virtual clinical sessions with changes in HRQoL. METHODS A total of 288 members completed the 4-item HRQoL measure at baseline and at 1 month following use of the Ginger on demand behavioral health platform. Baseline anxiety and depression levels were collected using the GAD-7 and PHQ-9, respectively. RESULTS Members completed on average 1.92 (SD 2.16) coaching sessions and 0.91 (SD 1.37) clinical sessions during the assessment month. Paired samples t tests revealed significant reductions in the average number of unhealthy mental health days between baseline (mean 16, SD 8.77 days) and follow-up (mean 13.2, SD 9.02 days; t287=5.73; P<.001), and in the average number of days adversely impacted (meanbaseline 10.9, meanfollow-up 8.19; t287=6.26; P<.001). Both subclinical members (t103=3.04; P=.003) and clinical members (t183=5.5; P<.001) demonstrated significant improvements through reductions in adversely impacted days over a month. Clinical members also demonstrated significant improvements through reductions in unhealthy mental health days (t183=5.82; P<.001). Finally, member engagement with virtual clinical sessions significantly predicted changes in unhealthy mental health days (B=–0.96; P=.04). CONCLUSIONS To our knowledge, this study is one of the first to use the HRQoL measure as an outcome in an evaluation of a digital behavioral health platform. Using real-world longitudinal data, our preliminary yet promising results show that short-term engagement with virtual care can be an effective means to improve HRQoL for members with subclinical and clinical symptoms. Further follow-up of reported HRQoL over several months is needed.
- Open Access
- 14 October 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Resilience in 2021—Descriptive Analysis of Individuals Accessing Virtual Mental Health Services: Retrospective Observational Study (Preprint)
BACKGROUND Psychological resilience has been extensively studied by developmental researchers, and there is a growing body of literature regarding its role in psychiatry and psychopathology research and practice. This study contributes to this growing literature by providing real-world evidence on the relationship between resilience and clinical symptoms among a large sample of employed Americans. OBJECTIVE This study aimed to describe resilience levels in individuals accessing Ginger, a virtual mental health system, in addition to the association of resilience with demographic characteristics, baseline depression, and anxiety symptoms. METHODS We conducted a retrospective observational study of 9165 members who signed up for Ginger and completed a baseline survey between January 1 and August 5, 2021. We used multivariate regression models to test for associations between baseline resilience and other member characteristics. RESULTS Baseline resilience scores centered on a mean of 23.84 (SD 6.56) and median of 24 (IQR 8) out of 40, with 81.0% (7424/9165) of the sample having low resilience at baseline. Despite having relatively higher resilience scores, members with no or mild depression or anxiety still had low resilience scores on average. Self-reported suicidal ideation was associated with lower resilience. CONCLUSIONS Overall, members had low baseline resilience, similar to resilience levels observed in trauma survivors in prior studies. Younger members and those with higher levels of depression and anxiety at intake reported lower levels of resilience at baseline. Notably, members with no or mild depression or anxiety still had low resilience scores on average, suggesting a need for mental health support among individuals who might not typically be recommended for treatment based on traditional clinical assessments, such as the 9-item Patient Health Questionnaire (PHQ-9) and the 7-item Generalized Anxiety Disorder scale (GAD-7). Two suggestions for topics of future research are to develop treatment recommendations based on the Connor-Davidson Resilience Scale and to understand the interaction between resilience levels and symptom-based outcome measures, such as the PHQ-9 and the GAD-7.
- 14 October 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessResilience in 2021—Descriptive Analysis of Individuals Accessing Virtual Mental Health Services: Retrospective Observational Study (Preprint)
BACKGROUND Psychological resilience has been extensively studied by developmental researchers, and there is a growing body of literature regarding its role in psychiatry and psychopathology research and practice. This study contributes to this growing literature by providing real-world evidence on the relationship between resilience and clinical symptoms among a large sample of employed Americans. OBJECTIVE This study aimed to describe resilience levels in individuals accessing Ginger, a virtual mental health system, in addition to the association of resilience with demographic characteristics, baseline depression, and anxiety symptoms. METHODS We conducted a retrospective observational study of 9165 members who signed up for Ginger and completed a baseline survey between January 1 and August 5, 2021. We used multivariate regression models to test for associations between baseline resilience and other member characteristics. RESULTS Baseline resilience scores centered on a mean of 23.84 (SD 6.56) and median of 24 (IQR 8) out of 40, with 81.0% (7424/9165) of the sample having low resilience at baseline. Despite having relatively higher resilience scores, members with no or mild depression or anxiety still had low resilience scores on average. Self-reported suicidal ideation was associated with lower resilience. CONCLUSIONS Overall, members had low baseline resilience, similar to resilience levels observed in trauma survivors in prior studies. Younger members and those with higher levels of depression and anxiety at intake reported lower levels of resilience at baseline. Notably, members with no or mild depression or anxiety still had low resilience scores on average, suggesting a need for mental health support among individuals who might not typically be recommended for treatment based on traditional clinical assessments, such as the 9-item Patient Health Questionnaire (PHQ-9) and the 7-item Generalized Anxiety Disorder scale (GAD-7). Two suggestions for topics of future research are to develop treatment recommendations based on the Connor-Davidson Resilience Scale and to understand the interaction between resilience levels and symptom-based outcome measures, such as the PHQ-9 and the GAD-7.