Brandon S. Aylward
Research Subjects
Research Interests
Brandon S. Aylward has a total of 33 co-authors. Here are the top co-authors by number of shared publications:
- Sarah Kunkle
Headspace, Santa Monica, CA, United States
7 shared publications - Grant Graziani
Ginger San Francisco, CA United States
6 shared publications - Emily Shih
Ginger San Francisco, CA United States
5 shared publications - Sharief Taraman
Eratos Therapeutics
2 shared publications Akhil Chaturvedi
1 shared publication- Akhil Chaturvedi
Headspace Health, San Francisco, CA, United States
1 shared publication
Latest Publications
- Open Access
- 19 January 2023
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 7, e38831
Content Recommendation Systems in Web-Based Mental Health Care: Real-world Application and Formative Evaluation
Background: Recommender systems have great potential in mental health care to personalize self-guided content for patients, allowing them to supplement their mental health treatment in a scalable way. Objective: In this paper, we describe and evaluate 2 knowledge-based content recommendation systems as parts of Ginger, an on-demand mental health platform, to bolster engagement in self-guided mental health content. Methods: We developed two algorithms to provide content recommendations in the Ginger mental health smartphone app: (1) one that uses users' responses to app onboarding questions to recommend content cards and (2) one that uses the semantic similarity between the transcript of a coaching conversation and the description of content cards to make recommendations after every session. As a measure of success for these recommendation algorithms, we examined the relevance of content cards to users’ conversations with their coach and completion rates of selected content within the app measured over 14,018 users. Results: In a real-world setting, content consumed in the recommendations section (or “Explore” in the app) had the highest completion rates (3353/7871, 42.6%) compared to other sections of the app, which had an average completion rate of 37.35% (21,982/58,614; P<.001). Within the app’s recommendations section, conversation-based content recommendations had 11.4% (1108/2364) higher completion rates per card than onboarding response-based recommendations (1712/4067; P=.003) and 26.1% higher than random recommendations (534/1440; P=.005). Studied via subject matter experts’ annotations, conversation-based recommendations had a 16.1% higher relevance rate for the top 5 recommended cards, averaged across sessions of varying lengths, compared to a random control (110 conversational sessions). Finally, it was observed that both age and gender variables were sensitive to different recommendation methods, with responsiveness to personalized recommendations being higher if the users were older than 35 years or identified as male. Conclusions: Recommender systems can help scale and supplement digital mental health care with personalized content and self-care recommendations. Onboarding-based recommendations are ideal for “cold starting” the process of recommending content for new users and users that tend to use the app just for content but not for therapy or coaching. The conversation-based recommendation algorithm allows for dynamic recommendations based on information gathered during coaching sessions, which is a critical capability, given the changing nature of mental health needs during treatment. The proposed algorithms are just one step toward the direction of outcome-driven personalization in mental health. Our future work will involve a robust causal evaluation of these algorithms using randomized controlled trials, along with consumer feedback–driven improvement of these algorithms, to drive better clinical outcomes.
- 19 January 2023
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 7, e38831
Open AccessContent Recommendation Systems in Web-Based Mental Health Care: Real-world Application and Formative Evaluation
Background: Recommender systems have great potential in mental health care to personalize self-guided content for patients, allowing them to supplement their mental health treatment in a scalable way. Objective: In this paper, we describe and evaluate 2 knowledge-based content recommendation systems as parts of Ginger, an on-demand mental health platform, to bolster engagement in self-guided mental health content. Methods: We developed two algorithms to provide content recommendations in the Ginger mental health smartphone app: (1) one that uses users' responses to app onboarding questions to recommend content cards and (2) one that uses the semantic similarity between the transcript of a coaching conversation and the description of content cards to make recommendations after every session. As a measure of success for these recommendation algorithms, we examined the relevance of content cards to users’ conversations with their coach and completion rates of selected content within the app measured over 14,018 users. Results: In a real-world setting, content consumed in the recommendations section (or “Explore” in the app) had the highest completion rates (3353/7871, 42.6%) compared to other sections of the app, which had an average completion rate of 37.35% (21,982/58,614; P<.001). Within the app’s recommendations section, conversation-based content recommendations had 11.4% (1108/2364) higher completion rates per card than onboarding response-based recommendations (1712/4067; P=.003) and 26.1% higher than random recommendations (534/1440; P=.005). Studied via subject matter experts’ annotations, conversation-based recommendations had a 16.1% higher relevance rate for the top 5 recommended cards, averaged across sessions of varying lengths, compared to a random control (110 conversational sessions). Finally, it was observed that both age and gender variables were sensitive to different recommendation methods, with responsiveness to personalized recommendations being higher if the users were older than 35 years or identified as male. Conclusions: Recommender systems can help scale and supplement digital mental health care with personalized content and self-care recommendations. Onboarding-based recommendations are ideal for “cold starting” the process of recommending content for new users and users that tend to use the app just for content but not for therapy or coaching. The conversation-based recommendation algorithm allows for dynamic recommendations based on information gathered during coaching sessions, which is a critical capability, given the changing nature of mental health needs during treatment. The proposed algorithms are just one step toward the direction of outcome-driven personalization in mental health. Our future work will involve a robust causal evaluation of these algorithms using randomized controlled trials, along with consumer feedback–driven improvement of these algorithms, to drive better clinical outcomes.
- 20 December 2022
- Published by SAGE Publications in Clinical Pediatrics
- Vol. 62 (7) , 673-677
Autism Spectrum Disorder: The New Asthma?
While autism spectrum disorder (ASD) has traditionally been diagnosed in specialty care, rising prevalence rates have strained specialist capacity and created significant diagnostic bottlenecks. With long wait times for specialist evaluations, pediatricians are increasingly being asked to play a greater role in identifying, diagnosing, and managing ASD within the medical home. In this commentary we draw parallels between the challenges pediatricians faced two decades ago learning how to manage pediatric asthma in the medical home, to those faced today in the field of primary care ASD management. Reflecting on the lessons learnt in primary care asthma management may help us construct a roadmap towards a higher and more consistent standard of ASD primary care for patients and their families.
- 20 December 2022
- Published by SAGE Publications in Clinical Pediatrics
- Vol. 62 (7) , 673-677
Autism Spectrum Disorder: The New Asthma?
While autism spectrum disorder (ASD) has traditionally been diagnosed in specialty care, rising prevalence rates have strained specialist capacity and created significant diagnostic bottlenecks. With long wait times for specialist evaluations, pediatricians are increasingly being asked to play a greater role in identifying, diagnosing, and managing ASD within the medical home. In this commentary we draw parallels between the challenges pediatricians faced two decades ago learning how to manage pediatric asthma in the medical home, to those faced today in the field of primary care ASD management. Reflecting on the lessons learnt in primary care asthma management may help us construct a roadmap towards a higher and more consistent standard of ASD primary care for patients and their families.
- Open Access
- 22 November 2022
- Published by Wolters Kluwer Health in Journal of Developmental & Behavioral Pediatrics
- Vol. 44 (2) , e126-e134
An Introduction to Artificial Intelligence in Developmental and Behavioral Pediatrics
Technological breakthroughs, together with the rapid growth of medical information and improved data connectivity, are creating dramatic shifts in the health care landscape, including the field of developmental and behavioral pediatrics. While medical information took an estimated 50 years to double in 1950, by 2020, it was projected to double every 73 days. Artificial intelligence (AI)–powered health technologies, once considered theoretical or research-exclusive concepts, are increasingly being granted regulatory approval and integrated into clinical care. In the United States, the Food and Drug Administration has cleared or approved over 160 health-related AI-based devices to date. These trends are only likely to accelerate as economic investment in AI health care outstrips investment in other sectors. The exponential increase in peer-reviewed AI-focused health care publications year over year highlights the speed of growth in this sector. As health care moves toward an era of intelligent technology powered by rich medical information, pediatricians will increasingly be asked to engage with tools and systems underpinned by AI. However, medical students and practicing clinicians receive insufficient training and lack preparedness for transitioning into a more AI-informed future. This article provides a brief primer on AI in health care. Underlying AI principles and key performance metrics are described, and the clinical potential of AI-driven technology together with potential pitfalls is explored within the developmental and behavioral pediatric health context.
- 22 November 2022
- Published by Wolters Kluwer Health in Journal of Developmental & Behavioral Pediatrics
- Vol. 44 (2) , e126-e134
Open AccessAn Introduction to Artificial Intelligence in Developmental and Behavioral Pediatrics
Technological breakthroughs, together with the rapid growth of medical information and improved data connectivity, are creating dramatic shifts in the health care landscape, including the field of developmental and behavioral pediatrics. While medical information took an estimated 50 years to double in 1950, by 2020, it was projected to double every 73 days. Artificial intelligence (AI)–powered health technologies, once considered theoretical or research-exclusive concepts, are increasingly being granted regulatory approval and integrated into clinical care. In the United States, the Food and Drug Administration has cleared or approved over 160 health-related AI-based devices to date. These trends are only likely to accelerate as economic investment in AI health care outstrips investment in other sectors. The exponential increase in peer-reviewed AI-focused health care publications year over year highlights the speed of growth in this sector. As health care moves toward an era of intelligent technology powered by rich medical information, pediatricians will increasingly be asked to engage with tools and systems underpinned by AI. However, medical students and practicing clinicians receive insufficient training and lack preparedness for transitioning into a more AI-informed future. This article provides a brief primer on AI in health care. Underlying AI principles and key performance metrics are described, and the clinical potential of AI-driven technology together with potential pitfalls is explored within the developmental and behavioral pediatric health context.
- Open Access
- 28 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e37169
Changes in Resilience Following Engagement With a Virtual Mental Health System: Real-world Observational Study
Background: Digital health services can serve as scalable solutions to address the growing demand for mental health care. However, more research is needed to better understand the association between engagement with care and improvements in subclinical outcomes. Objective: This study aims to fill this research gap by examining the relationship between members' engagement with the Ginger platform and changes in their psychological resilience. Methods: We conducted a retrospective observational study of 3272 members who accessed Ginger, an on-demand mental health service, between January 2021 and November 2021. Each member completed the 10-item Connor-Davidson Resilience Scale questionnaire, a measure of psychological resilience, at baseline and again during a 6- to 16-week follow-up window. Depression and anxiety symptoms (9-item Patient Health Questionnaire and 7-item Generalized Anxiety Disorder) were also measured. Linear regression was used to identify the association between engagement with Ginger's multiple care modalities and changes in resilience. Moderator analysis was conducted to test whether clinical depression or anxiety at baseline moderated the relationship between engagement level and changes in resilience. Results: Of the 3272 members, 2683 (82%) reported low resilience at baseline. The mean change in resilience was 0.77 (SD 5.50) points. Linear regression models showed that age and census region did not predict changes in resilience; however, male members showed larger improvements (coefficient=0.58; P=.04). Baseline mental health outcomes, including resilience and depression and anxiety symptoms, were strong predictors of changes in resilience. Every point decrease in baseline resilience is associated with a 0.28-point increase in change in resilience (P<.001), and members with no or mild depression and anxiety at baseline saw changes in resilience that were 1.44 points (P<.001) larger than their clinical counterparts. Engagement with the Ginger system predicted changes in resilience. Members who engaged with Ginger coaching, clinical services, or both improved their resilience by 1.82, 1.55, and 1.40 points, respectively (P<.001), more than those who only engaged with Ginger content. Screening negative for moderate to severe depression and anxiety at baseline was associated with larger improvements in resilience (coefficient=1.30; P<.001); however, subclinical status was not shown to be a moderator for the association between level of engagement and changes in resilience. Conclusions: Engagement with Ginger services was associated with improvements in resilience. Members who engaged in coaching or clinical care had significantly larger improvements compared with those who only engaged in self-guided content, regardless of whether a member screened positive for clinical depression or anxiety at baseline.
- 28 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e37169
Open AccessChanges in Resilience Following Engagement With a Virtual Mental Health System: Real-world Observational Study
Background: Digital health services can serve as scalable solutions to address the growing demand for mental health care. However, more research is needed to better understand the association between engagement with care and improvements in subclinical outcomes. Objective: This study aims to fill this research gap by examining the relationship between members' engagement with the Ginger platform and changes in their psychological resilience. Methods: We conducted a retrospective observational study of 3272 members who accessed Ginger, an on-demand mental health service, between January 2021 and November 2021. Each member completed the 10-item Connor-Davidson Resilience Scale questionnaire, a measure of psychological resilience, at baseline and again during a 6- to 16-week follow-up window. Depression and anxiety symptoms (9-item Patient Health Questionnaire and 7-item Generalized Anxiety Disorder) were also measured. Linear regression was used to identify the association between engagement with Ginger's multiple care modalities and changes in resilience. Moderator analysis was conducted to test whether clinical depression or anxiety at baseline moderated the relationship between engagement level and changes in resilience. Results: Of the 3272 members, 2683 (82%) reported low resilience at baseline. The mean change in resilience was 0.77 (SD 5.50) points. Linear regression models showed that age and census region did not predict changes in resilience; however, male members showed larger improvements (coefficient=0.58; P=.04). Baseline mental health outcomes, including resilience and depression and anxiety symptoms, were strong predictors of changes in resilience. Every point decrease in baseline resilience is associated with a 0.28-point increase in change in resilience (P<.001), and members with no or mild depression and anxiety at baseline saw changes in resilience that were 1.44 points (P<.001) larger than their clinical counterparts. Engagement with the Ginger system predicted changes in resilience. Members who engaged with Ginger coaching, clinical services, or both improved their resilience by 1.82, 1.55, and 1.40 points, respectively (P<.001), more than those who only engaged with Ginger content. Screening negative for moderate to severe depression and anxiety at baseline was associated with larger improvements in resilience (coefficient=1.30; P<.001); however, subclinical status was not shown to be a moderator for the association between level of engagement and changes in resilience. Conclusions: Engagement with Ginger services was associated with improvements in resilience. Members who engaged in coaching or clinical care had significantly larger improvements compared with those who only engaged in self-guided content, regardless of whether a member screened positive for clinical depression or anxiety at baseline.
- Open Access
- 22 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e36956
Association Between Care Modality and Use With Treatment Response Among Members Accessing Virtual Mental Health Services: Real-world Observational Study
Background: There is a growing bottleneck in mental health care, as the demand for services has outpaced the availability of mental health professionals. Consequently, many health systems have shifted to teletherapy as a scalable approach to increasing accessibility to care. Within these care models, various treatment modalities (eg, coaching and clinical care) are used to deliver support for anxiety and depression. However, more research is needed to better understand the differences in treatment responses. Objective: The purpose of this study was to examine the association between different care modalities and the levels of use with symptom score changes for members seeking virtual care services. Methods: We conducted an observational study of 4219 members who accessed Ginger, an on-demand mental health service, between September 2020 and September 2021. Using a mobile app, members can access text-based behavioral health coaching and virtual clinical services. This study focused on members with clinically elevated depression or anxiety levels at baseline. Logistic regressions were used to assess the association between care modalities and the levels of use with treatment response in depression and anxiety, using the Patient Health Questionnaire and Generalized Anxiety Disorder Assessment, respectively. Results: Of the 4219 members, 1623 (38.47%) demonstrated a full response to depression, and 1684 (39.91%) demonstrated a full response to anxiety. Members who completed care (ie, text-based coaching, virtual clinical therapy, hybrid of coaching, and clinical care) beyond the introductory session showed significantly increased odds of a full response compared with those who completed only limited care. Members who completed a hybrid of care had the highest odds of improvement; the odds of showing a full response in depression were 2.31 times higher (95% CI 1.91-2.80; P<.001) and in anxiety were 2.23 times higher (95% CI 1.84-2.70; P<.001) compared with members who completed limited care. For members who completed only coaching or clinical care, the largest effects were observed among those with high use. For members who completed a hybrid care program, we observed similar treatment responses across all levels of use. Conclusions: Our real-world study found that members who completed text-based coaching achieved full treatment responses at similar rates compared with members who completed virtual clinical care and members who completed a hybrid of care. There were no significant differences in the predicted probabilities of full treatment response between coaching and clinical care. Generally, the odds for a full response were highest among members with high use within each care modality; however, there were no differences in full-response treatment odds across levels of use with hybrid care. The results support the utility of digital behavioral health interventions and further highlight text-based coaching protocols as an accessible and suitable option when considering virtual care for treating anxiety and depression.
- 22 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e36956
Open AccessAssociation Between Care Modality and Use With Treatment Response Among Members Accessing Virtual Mental Health Services: Real-world Observational Study
Background: There is a growing bottleneck in mental health care, as the demand for services has outpaced the availability of mental health professionals. Consequently, many health systems have shifted to teletherapy as a scalable approach to increasing accessibility to care. Within these care models, various treatment modalities (eg, coaching and clinical care) are used to deliver support for anxiety and depression. However, more research is needed to better understand the differences in treatment responses. Objective: The purpose of this study was to examine the association between different care modalities and the levels of use with symptom score changes for members seeking virtual care services. Methods: We conducted an observational study of 4219 members who accessed Ginger, an on-demand mental health service, between September 2020 and September 2021. Using a mobile app, members can access text-based behavioral health coaching and virtual clinical services. This study focused on members with clinically elevated depression or anxiety levels at baseline. Logistic regressions were used to assess the association between care modalities and the levels of use with treatment response in depression and anxiety, using the Patient Health Questionnaire and Generalized Anxiety Disorder Assessment, respectively. Results: Of the 4219 members, 1623 (38.47%) demonstrated a full response to depression, and 1684 (39.91%) demonstrated a full response to anxiety. Members who completed care (ie, text-based coaching, virtual clinical therapy, hybrid of coaching, and clinical care) beyond the introductory session showed significantly increased odds of a full response compared with those who completed only limited care. Members who completed a hybrid of care had the highest odds of improvement; the odds of showing a full response in depression were 2.31 times higher (95% CI 1.91-2.80; P<.001) and in anxiety were 2.23 times higher (95% CI 1.84-2.70; P<.001) compared with members who completed limited care. For members who completed only coaching or clinical care, the largest effects were observed among those with high use. For members who completed a hybrid care program, we observed similar treatment responses across all levels of use. Conclusions: Our real-world study found that members who completed text-based coaching achieved full treatment responses at similar rates compared with members who completed virtual clinical care and members who completed a hybrid of care. There were no significant differences in the predicted probabilities of full treatment response between coaching and clinical care. Generally, the odds for a full response were highest among members with high use within each care modality; however, there were no differences in full-response treatment odds across levels of use with hybrid care. The results support the utility of digital behavioral health interventions and further highlight text-based coaching protocols as an accessible and suitable option when considering virtual care for treating anxiety and depression.
- Open Access
- 8 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e35352
Health-Related Quality of Life Among Members Using an On Demand Behavioral Health Platform: Pilot Observational Study
Background: Despite the well-known adverse health conditions and negative economic outcomes associated with mental health problems, accessing treatment is difficult due to reasons such as availability and cost. As a solution, digital mental health services have flooded the industry, and new studies are quickly emerging that support their potential as an accessible and cost-effective way to improve mental health outcomes. However, many mental health platforms typically use clinical tools such as the Patient Health Questionnaire-9 (PHQ-9) or General Anxiety Disorder-7 (GAD-7). Yet, many individuals that seek out care do not have clinical symptomatology and thus, traditional clinical measures may not adequately capture symptom improvement in general well-being. As an alternative, this study used the health-related quality of life (HRQoL) tool from the Centers for Disease Control and Prevention “Healthy Days” measure. This subjective measure of well-being is an effective way to capture HRQoL and might be better suited as an outcome measure for treatments that include both clinical and subclinical individuals. Objective: The purpose of this study was to describe changes in HRQoL in clinical and subclinical members assessing virtual care and to examine the association between text-based behavioral coaching and virtual clinical sessions with changes in HRQoL. Methods: A total of 288 members completed the 4-item HRQoL measure at baseline and at 1 month following use of the Ginger on demand behavioral health platform. Baseline anxiety and depression levels were collected using the GAD-7 and PHQ-9, respectively. Results: Members completed on average 1.92 (SD 2.16) coaching sessions and 0.91 (SD 1.37) clinical sessions during the assessment month. Paired samples t tests revealed significant reductions in the average number of unhealthy mental health days between baseline (mean 16, SD 8.77 days) and follow-up (mean 13.2, SD 9.02 days; t287=5.73; P<.001), and in the average number of days adversely impacted (meanbaseline 10.9, meanfollow-up 8.19; t287=6.26; P<.001). Both subclinical members (t103=3.04; P=.003) and clinical members (t183=5.5; P<.001) demonstrated significant improvements through reductions in adversely impacted days over a month. Clinical members also demonstrated significant improvements through reductions in unhealthy mental health days (t183=5.82; P<.001). Finally, member engagement with virtual clinical sessions significantly predicted changes in unhealthy mental health days (B=–0.96; P=.04). Conclusions: To our knowledge, this study is one of the first to use the HRQoL measure as an outcome in an evaluation of a digital behavioral health platform. Using real-world longitudinal data, our preliminary yet promising results show that short-term engagement with virtual care can be an effective means to improve HRQoL for members with subclinical and clinical symptoms. Further follow-up of reported HRQoL over several months is needed.
- 8 July 2022
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 6 (7) , e35352
Open AccessHealth-Related Quality of Life Among Members Using an On Demand Behavioral Health Platform: Pilot Observational Study
Background: Despite the well-known adverse health conditions and negative economic outcomes associated with mental health problems, accessing treatment is difficult due to reasons such as availability and cost. As a solution, digital mental health services have flooded the industry, and new studies are quickly emerging that support their potential as an accessible and cost-effective way to improve mental health outcomes. However, many mental health platforms typically use clinical tools such as the Patient Health Questionnaire-9 (PHQ-9) or General Anxiety Disorder-7 (GAD-7). Yet, many individuals that seek out care do not have clinical symptomatology and thus, traditional clinical measures may not adequately capture symptom improvement in general well-being. As an alternative, this study used the health-related quality of life (HRQoL) tool from the Centers for Disease Control and Prevention “Healthy Days” measure. This subjective measure of well-being is an effective way to capture HRQoL and might be better suited as an outcome measure for treatments that include both clinical and subclinical individuals. Objective: The purpose of this study was to describe changes in HRQoL in clinical and subclinical members assessing virtual care and to examine the association between text-based behavioral coaching and virtual clinical sessions with changes in HRQoL. Methods: A total of 288 members completed the 4-item HRQoL measure at baseline and at 1 month following use of the Ginger on demand behavioral health platform. Baseline anxiety and depression levels were collected using the GAD-7 and PHQ-9, respectively. Results: Members completed on average 1.92 (SD 2.16) coaching sessions and 0.91 (SD 1.37) clinical sessions during the assessment month. Paired samples t tests revealed significant reductions in the average number of unhealthy mental health days between baseline (mean 16, SD 8.77 days) and follow-up (mean 13.2, SD 9.02 days; t287=5.73; P<.001), and in the average number of days adversely impacted (meanbaseline 10.9, meanfollow-up 8.19; t287=6.26; P<.001). Both subclinical members (t103=3.04; P=.003) and clinical members (t183=5.5; P<.001) demonstrated significant improvements through reductions in adversely impacted days over a month. Clinical members also demonstrated significant improvements through reductions in unhealthy mental health days (t183=5.82; P<.001). Finally, member engagement with virtual clinical sessions significantly predicted changes in unhealthy mental health days (B=–0.96; P=.04). Conclusions: To our knowledge, this study is one of the first to use the HRQoL measure as an outcome in an evaluation of a digital behavioral health platform. Using real-world longitudinal data, our preliminary yet promising results show that short-term engagement with virtual care can be an effective means to improve HRQoL for members with subclinical and clinical symptoms. Further follow-up of reported HRQoL over several months is needed.
- Open Access
- 24 April 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Content recommendation systems in mental healthcare: Practical application and formative evaluation (Preprint)
BACKGROUND Recommender systems have great potential in mental health care to provide self-guided content to supplement the mental health journey for patients scalably; however, traditional filtering approaches often have skewed input data distributions, are static or may not account for changes in symptoms and clinical presentation. In this study, we describe and evaluate two knowledge-based content recommendation system models as part of Ginger, an on-demand mental health platform that seek to address the issues above. OBJECTIVE In this study, we describe and evaluate two knowledge-based content recommendation system models as part of Ginger, an on-demand mental health platform that seek to address the issues above. METHODS We developed two models to provide content recommendations for the Ginger mental health app content. First, a method that uses members' responses to onboarding questions to recommend content cards from the content library of the mental health platform. Second, a dynamic conversation-based recommendation method that matches the semantic similarity between the content of a conversation and the text description of content cards to make recommendations suitable for a conversational snippet. As a measure of success for these recommendation models, we examined the relevance of content cards to members’ conversations with their coach and completion rates of selected content within the app measured over 14018 users. RESULTS Conversation-based recommendation models performed better than random recommendations for all lengths of conversational sessions considering fractions of relevant cards as well as very relevant cards. In an offline analysis, conversation-based recommendations had a 16.1% higher relevance rate (for the top 5 recommended cards) averaged across sessions of varying lengths as compared to a random control algorithm. Comparing completion rates of content delivered in the app to over 14018 users, conversation-based content recommendations that had 11.4% higher completion rates per card than onboarding response based recommendations (P=.003) and 26.1% higher than random recommendations (P=.005)). CONCLUSIONS Recommender systems can help scale and supplement digital mental health care with relevant content and self-care recommendations. Conversation-based recommendation models allow for dynamic recommendations based on information gathered during the course of text-based coaching sessions, which is a critical capability given the changing nature of mental health over the course of treatment.
- 24 April 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessContent recommendation systems in mental healthcare: Practical application and formative evaluation (Preprint)
BACKGROUND Recommender systems have great potential in mental health care to provide self-guided content to supplement the mental health journey for patients scalably; however, traditional filtering approaches often have skewed input data distributions, are static or may not account for changes in symptoms and clinical presentation. In this study, we describe and evaluate two knowledge-based content recommendation system models as part of Ginger, an on-demand mental health platform that seek to address the issues above. OBJECTIVE In this study, we describe and evaluate two knowledge-based content recommendation system models as part of Ginger, an on-demand mental health platform that seek to address the issues above. METHODS We developed two models to provide content recommendations for the Ginger mental health app content. First, a method that uses members' responses to onboarding questions to recommend content cards from the content library of the mental health platform. Second, a dynamic conversation-based recommendation method that matches the semantic similarity between the content of a conversation and the text description of content cards to make recommendations suitable for a conversational snippet. As a measure of success for these recommendation models, we examined the relevance of content cards to members’ conversations with their coach and completion rates of selected content within the app measured over 14018 users. RESULTS Conversation-based recommendation models performed better than random recommendations for all lengths of conversational sessions considering fractions of relevant cards as well as very relevant cards. In an offline analysis, conversation-based recommendations had a 16.1% higher relevance rate (for the top 5 recommended cards) averaged across sessions of varying lengths as compared to a random control algorithm. Comparing completion rates of content delivered in the app to over 14018 users, conversation-based content recommendations that had 11.4% higher completion rates per card than onboarding response based recommendations (P=.003) and 26.1% higher than random recommendations (P=.005)). CONCLUSIONS Recommender systems can help scale and supplement digital mental health care with relevant content and self-care recommendations. Conversation-based recommendation models allow for dynamic recommendations based on information gathered during the course of text-based coaching sessions, which is a critical capability given the changing nature of mental health over the course of treatment.
- Open Access
- 1 February 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Association Between Care Modality and Use With Treatment Response Among Members Accessing Virtual Mental Health Services: Real-world Observational Study (Preprint)
BACKGROUND There is a growing bottleneck in mental health care, as the demand for services has outpaced the availability of mental health professionals. Consequently, many health systems have shifted to teletherapy as a scalable approach to increasing accessibility to care. Within these care models, various treatment modalities (eg, coaching and clinical care) are used to deliver support for anxiety and depression. However, more research is needed to better understand the differences in treatment responses. OBJECTIVE The purpose of this study was to examine the association between different care modalities and the levels of use with symptom score changes for members seeking virtual care services. METHODS We conducted an observational study of 4219 members who accessed Ginger, an on-demand mental health service, between September 2020 and September 2021. Using a mobile app, members can access text-based behavioral health coaching and virtual clinical services. This study focused on members with clinically elevated depression or anxiety levels at baseline. Logistic regressions were used to assess the association between care modalities and the levels of use with treatment response in depression and anxiety, using the Patient Health Questionnaire and Generalized Anxiety Disorder Assessment, respectively. RESULTS Of the 4219 members, 1623 (38.47%) demonstrated a full response to depression, and 1684 (39.91%) demonstrated a full response to anxiety. Members who completed care (ie, text-based coaching, virtual clinical therapy, hybrid of coaching, and clinical care) beyond the introductory session showed significantly increased odds of a full response compared with those who completed only limited care. Members who completed a hybrid of care had the highest odds of improvement; the odds of showing a full response in depression were 2.31 times higher (95% CI 1.91-2.80; P<.001) and in anxiety were 2.23 times higher (95% CI 1.84-2.70; P<.001) compared with members who completed limited care. For members who completed only coaching or clinical care, the largest effects were observed among those with high use. For members who completed a hybrid care program, we observed similar treatment responses across all levels of use. CONCLUSIONS Our real-world study found that members who completed text-based coaching achieved full treatment responses at similar rates compared with members who completed virtual clinical care and members who completed a hybrid of care. There were no significant differences in the predicted probabilities of full treatment response between coaching and clinical care. Generally, the odds for a full response were highest among members with high use within each care modality; however, there were no differences in full-response treatment odds across levels of use with hybrid care. The results support the utility of digital behavioral health interventions and further highlight text-based coaching protocols as an accessible and suitable option when considering virtual care for treating anxiety and depression.
- 1 February 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessAssociation Between Care Modality and Use With Treatment Response Among Members Accessing Virtual Mental Health Services: Real-world Observational Study (Preprint)
BACKGROUND There is a growing bottleneck in mental health care, as the demand for services has outpaced the availability of mental health professionals. Consequently, many health systems have shifted to teletherapy as a scalable approach to increasing accessibility to care. Within these care models, various treatment modalities (eg, coaching and clinical care) are used to deliver support for anxiety and depression. However, more research is needed to better understand the differences in treatment responses. OBJECTIVE The purpose of this study was to examine the association between different care modalities and the levels of use with symptom score changes for members seeking virtual care services. METHODS We conducted an observational study of 4219 members who accessed Ginger, an on-demand mental health service, between September 2020 and September 2021. Using a mobile app, members can access text-based behavioral health coaching and virtual clinical services. This study focused on members with clinically elevated depression or anxiety levels at baseline. Logistic regressions were used to assess the association between care modalities and the levels of use with treatment response in depression and anxiety, using the Patient Health Questionnaire and Generalized Anxiety Disorder Assessment, respectively. RESULTS Of the 4219 members, 1623 (38.47%) demonstrated a full response to depression, and 1684 (39.91%) demonstrated a full response to anxiety. Members who completed care (ie, text-based coaching, virtual clinical therapy, hybrid of coaching, and clinical care) beyond the introductory session showed significantly increased odds of a full response compared with those who completed only limited care. Members who completed a hybrid of care had the highest odds of improvement; the odds of showing a full response in depression were 2.31 times higher (95% CI 1.91-2.80; P<.001) and in anxiety were 2.23 times higher (95% CI 1.84-2.70; P<.001) compared with members who completed limited care. For members who completed only coaching or clinical care, the largest effects were observed among those with high use. For members who completed a hybrid care program, we observed similar treatment responses across all levels of use. CONCLUSIONS Our real-world study found that members who completed text-based coaching achieved full treatment responses at similar rates compared with members who completed virtual clinical care and members who completed a hybrid of care. There were no significant differences in the predicted probabilities of full treatment response between coaching and clinical care. Generally, the odds for a full response were highest among members with high use within each care modality; however, there were no differences in full-response treatment odds across levels of use with hybrid care. The results support the utility of digital behavioral health interventions and further highlight text-based coaching protocols as an accessible and suitable option when considering virtual care for treating anxiety and depression.
- Open Access
- 1 December 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Health-Related Quality of Life Among Members Using an On Demand Behavioral Health Platform: Pilot Observational Study (Preprint)
BACKGROUND Despite the well-known adverse health conditions and negative economic outcomes associated with mental health problems, accessing treatment is difficult due to reasons such as availability and cost. As a solution, digital mental health services have flooded the industry, and new studies are quickly emerging that support their potential as an accessible and cost-effective way to improve mental health outcomes. However, many mental health platforms typically use clinical tools such as the Patient Health Questionnaire-9 (PHQ-9) or General Anxiety Disorder-7 (GAD-7). Yet, many individuals that seek out care do not have clinical symptomatology and thus, traditional clinical measures may not adequately capture symptom improvement in general well-being. As an alternative, this study used the health-related quality of life (HRQoL) tool from the Centers for Disease Control and Prevention “Healthy Days” measure. This subjective measure of well-being is an effective way to capture HRQoL and might be better suited as an outcome measure for treatments that include both clinical and subclinical individuals. OBJECTIVE The purpose of this study was to describe changes in HRQoL in clinical and subclinical members assessing virtual care and to examine the association between text-based behavioral coaching and virtual clinical sessions with changes in HRQoL. METHODS A total of 288 members completed the 4-item HRQoL measure at baseline and at 1 month following use of the Ginger on demand behavioral health platform. Baseline anxiety and depression levels were collected using the GAD-7 and PHQ-9, respectively. RESULTS Members completed on average 1.92 (SD 2.16) coaching sessions and 0.91 (SD 1.37) clinical sessions during the assessment month. Paired samples t tests revealed significant reductions in the average number of unhealthy mental health days between baseline (mean 16, SD 8.77 days) and follow-up (mean 13.2, SD 9.02 days; t287=5.73; P<.001), and in the average number of days adversely impacted (meanbaseline 10.9, meanfollow-up 8.19; t287=6.26; P<.001). Both subclinical members (t103=3.04; P=.003) and clinical members (t183=5.5; P<.001) demonstrated significant improvements through reductions in adversely impacted days over a month. Clinical members also demonstrated significant improvements through reductions in unhealthy mental health days (t183=5.82; P<.001). Finally, member engagement with virtual clinical sessions significantly predicted changes in unhealthy mental health days (B=–0.96; P=.04). CONCLUSIONS To our knowledge, this study is one of the first to use the HRQoL measure as an outcome in an evaluation of a digital behavioral health platform. Using real-world longitudinal data, our preliminary yet promising results show that short-term engagement with virtual care can be an effective means to improve HRQoL for members with subclinical and clinical symptoms. Further follow-up of reported HRQoL over several months is needed.
- 1 December 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessHealth-Related Quality of Life Among Members Using an On Demand Behavioral Health Platform: Pilot Observational Study (Preprint)
BACKGROUND Despite the well-known adverse health conditions and negative economic outcomes associated with mental health problems, accessing treatment is difficult due to reasons such as availability and cost. As a solution, digital mental health services have flooded the industry, and new studies are quickly emerging that support their potential as an accessible and cost-effective way to improve mental health outcomes. However, many mental health platforms typically use clinical tools such as the Patient Health Questionnaire-9 (PHQ-9) or General Anxiety Disorder-7 (GAD-7). Yet, many individuals that seek out care do not have clinical symptomatology and thus, traditional clinical measures may not adequately capture symptom improvement in general well-being. As an alternative, this study used the health-related quality of life (HRQoL) tool from the Centers for Disease Control and Prevention “Healthy Days” measure. This subjective measure of well-being is an effective way to capture HRQoL and might be better suited as an outcome measure for treatments that include both clinical and subclinical individuals. OBJECTIVE The purpose of this study was to describe changes in HRQoL in clinical and subclinical members assessing virtual care and to examine the association between text-based behavioral coaching and virtual clinical sessions with changes in HRQoL. METHODS A total of 288 members completed the 4-item HRQoL measure at baseline and at 1 month following use of the Ginger on demand behavioral health platform. Baseline anxiety and depression levels were collected using the GAD-7 and PHQ-9, respectively. RESULTS Members completed on average 1.92 (SD 2.16) coaching sessions and 0.91 (SD 1.37) clinical sessions during the assessment month. Paired samples t tests revealed significant reductions in the average number of unhealthy mental health days between baseline (mean 16, SD 8.77 days) and follow-up (mean 13.2, SD 9.02 days; t287=5.73; P<.001), and in the average number of days adversely impacted (meanbaseline 10.9, meanfollow-up 8.19; t287=6.26; P<.001). Both subclinical members (t103=3.04; P=.003) and clinical members (t183=5.5; P<.001) demonstrated significant improvements through reductions in adversely impacted days over a month. Clinical members also demonstrated significant improvements through reductions in unhealthy mental health days (t183=5.82; P<.001). Finally, member engagement with virtual clinical sessions significantly predicted changes in unhealthy mental health days (B=–0.96; P=.04). CONCLUSIONS To our knowledge, this study is one of the first to use the HRQoL measure as an outcome in an evaluation of a digital behavioral health platform. Using real-world longitudinal data, our preliminary yet promising results show that short-term engagement with virtual care can be an effective means to improve HRQoL for members with subclinical and clinical symptoms. Further follow-up of reported HRQoL over several months is needed.
- Top Cited PapersOpen Access
- 10 September 2021
- Published by Wolters Kluwer Health in Journal of Developmental & Behavioral Pediatrics
- Vol. 42 (8) , 682-689
Racial, Ethnic, and Sociodemographic Disparities in Diagnosis of Children with Autism Spectrum Disorder
This special article uses a biosocial-ecological framework to discuss findings in the literature on racial, ethnic, and sociodemographic diagnostic disparities in autism spectrum disorder. We draw explanations from this framework on the complex and cumulative influences of social injustices across interpersonal and systemic levels.
- 10 September 2021
- Published by Wolters Kluwer Health in Journal of Developmental & Behavioral Pediatrics
- Vol. 42 (8) , 682-689
Top Cited PapersOpen AccessRacial, Ethnic, and Sociodemographic Disparities in Diagnosis of Children with Autism Spectrum Disorder
This special article uses a biosocial-ecological framework to discuss findings in the literature on racial, ethnic, and sociodemographic diagnostic disparities in autism spectrum disorder. We draw explanations from this framework on the complex and cumulative influences of social injustices across interpersonal and systemic levels.