Kurtis Stewart
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Kurtis Stewart has a total of 122 co-authors. Here are the top co-authors by number of shared publications:
- Kendall Ho
Department of Emergency Medicine, University of British Columbia, Vancouver, BC, Canada
10 shared publications - Helen Novak Lauscher
Department of Emergency Medicine, University of British Columbia, Vancouver, BC, Canada
9 shared publications - Khalida Ismail
Department of Psychological Medicine, Diabetes, Psychology and Psychiatry Research Group, King’s College London, London, UK
6 shared publications Kendall Ho
5 shared publications- Erika Pritchard
Rural Coordination Centre of British Columbia
4 shared publications - John Pawlovich
Department of Family Practice, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
4 shared publications
Latest Publications
- Open Access
- 1 October 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Patient concordance following an urgent virtual physician consultation in a provincial triage program in British Columbia, Canada (Preprint)
BACKGROUND Virtual nurse-physician collaborations in pre-hospital triage program such as HealthLink BC’s 8-1-1 service can reduce unnecessary emergency department (ED) by referring patients to the appropriate level of follow-up care, ranging from “Go to ED Now” to “Home Treatment”. However, limited evidence exists on the level to which patients follow through with the advice provided. OBJECTIVE We sought to describe the patient-level and geographic factors associated with caller concordance to 8-1-1 physician advice and develop predictive machine learning (ML) models of concordance. METHODS All calls to the 8-1-1 virtual physician service made between April 6, 2020 and March 31, 2023 with a follow-up care recommendation were eligible. Caller records were linked to provincial health administrative databases to determine demographic and pre-/post-call health service use information and geographic information based on health or census area. Caller concordance was calculated by physician discharge disposition. Descriptive analyses summarized concordance rates by demographic characteristics and health regions. ML models, including support vector machines, neural networks, and extreme gradient boosting (XGBoost), were developed using over 380 preprocessed features to predict concordance and evaluated using F1-scores. RESULTS 113,090 calls were included in the analysis. Concordance rates varied by physician disposition: they were highest for “Go to ED Now” calls (61.3%). When stratified, concordance rates were most strongly associated with health region and rurality. The XGBoost model had the strongest predictive performance for the “Go to ED Now” disposition (F1-score=0.78), while other models and other dispositions had F1-scores<=0.63. Health region, rurality, sex, and primary care attachment were consistently the strongest features influencing model predictions. In a sensitivity analysis expanding the concordance definitions, XGBoost models’ F1-scores increased up to 0.90. CONCLUSIONS Caller concordance with virtual physician advice was highest for the most urgent care recommendation, but varied greatly by health region, indicating significant disparities for rural patients trying to access immediate ED care. XGBoost models had strong predictive performance for more urgent care recommendations, especially when considering concordance over longer timeframes. ML models could be leveraged to support patients at risk of discordance and subsequent potentially negative outcomes by personalizing virtual triage interventions.
- 1 October 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessPatient concordance following an urgent virtual physician consultation in a provincial triage program in British Columbia, Canada (Preprint)
BACKGROUND Virtual nurse-physician collaborations in pre-hospital triage program such as HealthLink BC’s 8-1-1 service can reduce unnecessary emergency department (ED) by referring patients to the appropriate level of follow-up care, ranging from “Go to ED Now” to “Home Treatment”. However, limited evidence exists on the level to which patients follow through with the advice provided. OBJECTIVE We sought to describe the patient-level and geographic factors associated with caller concordance to 8-1-1 physician advice and develop predictive machine learning (ML) models of concordance. METHODS All calls to the 8-1-1 virtual physician service made between April 6, 2020 and March 31, 2023 with a follow-up care recommendation were eligible. Caller records were linked to provincial health administrative databases to determine demographic and pre-/post-call health service use information and geographic information based on health or census area. Caller concordance was calculated by physician discharge disposition. Descriptive analyses summarized concordance rates by demographic characteristics and health regions. ML models, including support vector machines, neural networks, and extreme gradient boosting (XGBoost), were developed using over 380 preprocessed features to predict concordance and evaluated using F1-scores. RESULTS 113,090 calls were included in the analysis. Concordance rates varied by physician disposition: they were highest for “Go to ED Now” calls (61.3%). When stratified, concordance rates were most strongly associated with health region and rurality. The XGBoost model had the strongest predictive performance for the “Go to ED Now” disposition (F1-score=0.78), while other models and other dispositions had F1-scores<=0.63. Health region, rurality, sex, and primary care attachment were consistently the strongest features influencing model predictions. In a sensitivity analysis expanding the concordance definitions, XGBoost models’ F1-scores increased up to 0.90. CONCLUSIONS Caller concordance with virtual physician advice was highest for the most urgent care recommendation, but varied greatly by health region, indicating significant disparities for rural patients trying to access immediate ED care. XGBoost models had strong predictive performance for more urgent care recommendations, especially when considering concordance over longer timeframes. ML models could be leveraged to support patients at risk of discordance and subsequent potentially negative outcomes by personalizing virtual triage interventions.
- Open Access
- 6 February 2026
- Published by SAGE Publications in Healthcare Management Forum
- Vol. 39 (3) , 276-284
Healthcare Providers’ Experiences Accessing Real-Time Virtual Support: Informing More Equitable and Inclusive Healthcare Access in British Columbia’s Rural, Remote, First Nations, and Other Indigenous Peoples and Communities
Healthcare Providers (HCPs) serving Rural, Remote, First Nations, and other Indigenous (RRFNI) communities face unique challenges in delivering longitudinal care due to geographic isolation. The Real-Time Virtual Support Services (RTVS) network aims to improve equitable access to healthcare and provide collegial support for HCPs in RRFNI communities across British Columbia. The objective of this study was to understand HCPs’ experiences with RTVS and identify improvement areas. Data were collected through semi-structured interviews with HCPs that were recorded, transcribed, and openly coded. Twenty HCPs using RTVS were interviewed during 2022-2023. The constant comparative method was used to develop themes. Themes focused on RTVS’s benefits and outcomes including increased clinical confidence, reduced provider anxiety, respectful and collegial support, reduced administrative burden, and recruitment and retention support. Challenges included occasional service disruptions and limited Wi-Fi availability. These findings provide in-depth and contextualized feedback informing the development of RTVS.
- 6 February 2026
- Published by SAGE Publications in Healthcare Management Forum
- Vol. 39 (3) , 276-284
Open AccessHealthcare Providers’ Experiences Accessing Real-Time Virtual Support: Informing More Equitable and Inclusive Healthcare Access in British Columbia’s Rural, Remote, First Nations, and Other Indigenous Peoples and Communities
Healthcare Providers (HCPs) serving Rural, Remote, First Nations, and other Indigenous (RRFNI) communities face unique challenges in delivering longitudinal care due to geographic isolation. The Real-Time Virtual Support Services (RTVS) network aims to improve equitable access to healthcare and provide collegial support for HCPs in RRFNI communities across British Columbia. The objective of this study was to understand HCPs’ experiences with RTVS and identify improvement areas. Data were collected through semi-structured interviews with HCPs that were recorded, transcribed, and openly coded. Twenty HCPs using RTVS were interviewed during 2022-2023. The constant comparative method was used to develop themes. Themes focused on RTVS’s benefits and outcomes including increased clinical confidence, reduced provider anxiety, respectful and collegial support, reduced administrative burden, and recruitment and retention support. Challenges included occasional service disruptions and limited Wi-Fi availability. These findings provide in-depth and contextualized feedback informing the development of RTVS.
- 15 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 31 (12) , 10384-10402
VIVA: Virtual Healthcare Interactions Using Visual Analytics, With Controllability Through Configuration
At the beginning of the COVID-19 pandemic, HealthLink BC (HLBC) rapidly integrated physicians into the triage process of their virtual healthcare service to improve patient outcomes and satisfaction with this service and preserve health care system capacity. We present the design and implementation of a visual analytics tool, VIVA (Virtual healthcare Interactions using Visual Analytics), to support HLBC in analysing various forms of usage data from the service. We abstract HLBC’s data and data analysis tasks, which we use to inform our design of VIVA. We also present the interactive workflow abstraction of Scan, Act, Adapt. We validate VIVA’s design through three case studies with stakeholder domain experts. We also propose the Controllability Through Configuration model to conduct and analyze design studies, and discuss architectural evolution of VIVA through that lens. It articulates configuration, both that specified by a developer or technical power user and that constructed automatically through log data from previous interactive sessions, as a bridge between the rigidity of hardwired programming and the time-consuming implementation of full end-user interactivity.
- 15 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 31 (12) , 10384-10402
VIVA: Virtual Healthcare Interactions Using Visual Analytics, With Controllability Through Configuration
At the beginning of the COVID-19 pandemic, HealthLink BC (HLBC) rapidly integrated physicians into the triage process of their virtual healthcare service to improve patient outcomes and satisfaction with this service and preserve health care system capacity. We present the design and implementation of a visual analytics tool, VIVA (Virtual healthcare Interactions using Visual Analytics), to support HLBC in analysing various forms of usage data from the service. We abstract HLBC’s data and data analysis tasks, which we use to inform our design of VIVA. We also present the interactive workflow abstraction of Scan, Act, Adapt. We validate VIVA’s design through three case studies with stakeholder domain experts. We also propose the Controllability Through Configuration model to conduct and analyze design studies, and discuss architectural evolution of VIVA through that lens. It articulates configuration, both that specified by a developer or technical power user and that constructed automatically through log data from previous interactive sessions, as a bridge between the rigidity of hardwired programming and the time-consuming implementation of full end-user interactivity.
- Open Access
VIVA: Virtual Healthcare Interactions Using Visual Analytics, With Controllability Through Configuration
At the beginning of the COVID-19 pandemic, HealthLink BC (HLBC) rapidly integrated physicians into the triage process of their virtual healthcare service to improve patient outcomes and satisfaction with this service and preserve health care system capacity. We present the design and implementation of a visual analytics tool, VIVA (Virtual healthcare Interactions using Visual Analytics), to support HLBC in analysing various forms of usage data from the service. We abstract HLBC's data and data analysis tasks, which we use to inform our design of VIVA. We also present the interactive workflow abstraction of Scan, Act, Adapt. We validate VIVA's design through three case studies with stakeholder domain experts. We also propose the Controllability Through Configuration model to conduct and analyze design studies, and discuss architectural evolution of VIVA through that lens. It articulates configuration, both that specified by a developer or technical power user and that constructed automatically through log data from previous interactive sessions, as a bridge between the rigidity of hardwired programming and the time-consuming implementation of full end-user interactivity. Availability: Supplemental materials at https://osf.io/wv38n
Open AccessVIVA: Virtual Healthcare Interactions Using Visual Analytics, With Controllability Through Configuration
At the beginning of the COVID-19 pandemic, HealthLink BC (HLBC) rapidly integrated physicians into the triage process of their virtual healthcare service to improve patient outcomes and satisfaction with this service and preserve health care system capacity. We present the design and implementation of a visual analytics tool, VIVA (Virtual healthcare Interactions using Visual Analytics), to support HLBC in analysing various forms of usage data from the service. We abstract HLBC's data and data analysis tasks, which we use to inform our design of VIVA. We also present the interactive workflow abstraction of Scan, Act, Adapt. We validate VIVA's design through three case studies with stakeholder domain experts. We also propose the Controllability Through Configuration model to conduct and analyze design studies, and discuss architectural evolution of VIVA through that lens. It articulates configuration, both that specified by a developer or technical power user and that constructed automatically through log data from previous interactive sessions, as a bridge between the rigidity of hardwired programming and the time-consuming implementation of full end-user interactivity. Availability: Supplemental materials at https://osf.io/wv38n
- Open Access
- 9 July 2025
- Published by SAGE Publications in Healthcare Management Forum
- Vol. 38 (5) , 483-489
Physicians’ experiences delivering provincial real-time virtual support services: A qualitative interview study
Rural, remote, and Indigenous communities in British Columbia (BC) tend to have lower access to healthcare providers and poorer health outcomes—an inequality that the COVID-19 pandemic has exacerbated. In response, Real-Time Virtual Support (RTVS) pathways were developed to advance equitable access to care for patients and provide peer support to physicians working in underserved communities. This study aimed to describe the perspectives of Virtual Physicians (VPs) who delivered the RTVS services. Forty-five RTVS VPs engaged in 30-minute semi-structured interviews about their experiences and perspectives delivering RTVS. Three themes emerged: (1) RTVS’s contributions to VPs’ personal and professional development; (2) impacts on communities; and (3) considerations for the availability and expansion. VPs identified incremental expansion and attaining funding stability as critical next steps for virtual healthcare in BC. This evidence informed RTVS program evaluation and may provide learnings relevant to other jurisdictions.
- 9 July 2025
- Published by SAGE Publications in Healthcare Management Forum
- Vol. 38 (5) , 483-489
Open AccessPhysicians’ experiences delivering provincial real-time virtual support services: A qualitative interview study
Rural, remote, and Indigenous communities in British Columbia (BC) tend to have lower access to healthcare providers and poorer health outcomes—an inequality that the COVID-19 pandemic has exacerbated. In response, Real-Time Virtual Support (RTVS) pathways were developed to advance equitable access to care for patients and provide peer support to physicians working in underserved communities. This study aimed to describe the perspectives of Virtual Physicians (VPs) who delivered the RTVS services. Forty-five RTVS VPs engaged in 30-minute semi-structured interviews about their experiences and perspectives delivering RTVS. Three themes emerged: (1) RTVS’s contributions to VPs’ personal and professional development; (2) impacts on communities; and (3) considerations for the availability and expansion. VPs identified incremental expansion and attaining funding stability as critical next steps for virtual healthcare in BC. This evidence informed RTVS program evaluation and may provide learnings relevant to other jurisdictions.
- Open Access
- 19 February 2025
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27, e56766
Estimating Patient and Family Costs and CO2 Emissions for Telehealth and In-Person Health Care Appointments in British Columbia, Canada: Geospatial Mixed Methods Study
Background: Patients inevitably incur some cost for accessing health care, even in universal systems such as Canada. The COVID-19 pandemic dramatically shifted health care delivery from in-person to telehealth services, also shifting the proportion of costs offset by patients and their families by reducing the need to travel to in-person appointments. Objective: This study aimed to develop a method for estimating the costs patients and their families incur and CO2 emissions attributed to travel needed for emergency department (ED) visits, hospitalizations, and physician appointments. Methods: We present a method to evaluate the costs associated with in-person and telehealth care appointments from the perspective of patients, their families, and the environment. We used ED locations, road distances, and duration of appointment to account for costs paid by patients (ie, lost productivity, informal caregiving, and out-of-pocket expenses) attributed to travel to receive medical care. Costs to the environment were evaluated by calculating the amount of CO2 emitted per medical visit. Using our costs calculated per visit, we apply our method to calculate total patient costs for a simulated population over 1 year. Results: Our method estimates that patients in British Columbia pay up to $300 (2023 CAD, CAD $1=US $0.86) on average to attend an in-person ED visit, depending on where they live; $166 may be attributed to lost productivity, $83 to informal caregiving, and $50 to out-of-pocket expenses. These estimates are higher than most observed cost estimates. In addition, avoiding in-person care diverts up to 13 kg of CO2 per medical visit, depending on the distance and frequency of travel to appointments. This translates to up to $0.70 in carbon costs per visit, or cumulatively $44,120 per year in British Columbia, conventionally not included in patient cost estimates. Conclusions: We present a novel method for estimating patient-incurred costs and CO2 emissions from accessing health care and apply it to estimate that every year, patients in British Columbia pay upwards of 30 million dollars to access health care services, primarily for medical travel. Our method adds to the economic evaluation literature by providing a more comprehensive and context-modifiable calculation of patient costs that will allow for more informed decision-making regarding health care services.
- 19 February 2025
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27, e56766
Open AccessEstimating Patient and Family Costs and CO2 Emissions for Telehealth and In-Person Health Care Appointments in British Columbia, Canada: Geospatial Mixed Methods Study
Background: Patients inevitably incur some cost for accessing health care, even in universal systems such as Canada. The COVID-19 pandemic dramatically shifted health care delivery from in-person to telehealth services, also shifting the proportion of costs offset by patients and their families by reducing the need to travel to in-person appointments. Objective: This study aimed to develop a method for estimating the costs patients and their families incur and CO2 emissions attributed to travel needed for emergency department (ED) visits, hospitalizations, and physician appointments. Methods: We present a method to evaluate the costs associated with in-person and telehealth care appointments from the perspective of patients, their families, and the environment. We used ED locations, road distances, and duration of appointment to account for costs paid by patients (ie, lost productivity, informal caregiving, and out-of-pocket expenses) attributed to travel to receive medical care. Costs to the environment were evaluated by calculating the amount of CO2 emitted per medical visit. Using our costs calculated per visit, we apply our method to calculate total patient costs for a simulated population over 1 year. Results: Our method estimates that patients in British Columbia pay up to $300 (2023 CAD, CAD $1=US $0.86) on average to attend an in-person ED visit, depending on where they live; $166 may be attributed to lost productivity, $83 to informal caregiving, and $50 to out-of-pocket expenses. These estimates are higher than most observed cost estimates. In addition, avoiding in-person care diverts up to 13 kg of CO2 per medical visit, depending on the distance and frequency of travel to appointments. This translates to up to $0.70 in carbon costs per visit, or cumulatively $44,120 per year in British Columbia, conventionally not included in patient cost estimates. Conclusions: We present a novel method for estimating patient-incurred costs and CO2 emissions from accessing health care and apply it to estimate that every year, patients in British Columbia pay upwards of 30 million dollars to access health care services, primarily for medical travel. Our method adds to the economic evaluation literature by providing a more comprehensive and context-modifiable calculation of patient costs that will allow for more informed decision-making regarding health care services.
- Open Access
- 11 September 2024
- Published by JMIR Publications Inc. in Interactive Journal of Medical Research
- Vol. 13 (1) , e56729
Improving Triage Accuracy in Prehospital Emergency Telemedicine: Scoping Review of Machine Learning–Enhanced Approaches
Background: Prehospital telemedicine triage systems combined with machine learning (ML) methods have the potential to improve triage accuracy and safely redirect low-acuity patients from attending the emergency department. However, research in prehospital settings is limited but needed; emergency department overcrowding and adverse patient outcomes are increasingly common. Objective: In this scoping review, we sought to characterize the existing methods for ML-enhanced telemedicine emergency triage. In order to support future research, we aimed to delineate what data sources, predictors, labels, ML models, and performance metrics were used, and in which telemedicine triage systems these methods were applied. Methods: A scoping review was conducted, querying multiple databases (MEDLINE, PubMed, Scopus, and IEEE Xplore) through February 24, 2023, to identify potential ML-enhanced methods, and for those eligible, relevant study characteristics were extracted, including prehospital triage setting, types of predictors, ground truth labeling method, ML models used, and performance metrics. Inclusion criteria were restricted to the triage of emergency telemedicine services using ML methods on an undifferentiated (disease nonspecific) population. Only primary research studies in English were considered. Furthermore, only those studies using data collected remotely (as opposed to derived from physical assessments) were included. In order to limit bias, we exclusively included articles identified through our predefined search criteria and had 3 researchers (DR, JS, and KS) independently screen the resulting studies. We conducted a narrative synthesis of findings to establish a knowledge base in this domain and identify potential gaps to be addressed in forthcoming ML-enhanced methods. Results: A total of 165 unique records were screened for eligibility and 15 were included in the review. Most studies applied ML methods during emergency medical dispatch (7/15, 47%) or used chatbot applications (5/15, 33%). Patient demographics and health status variables were the most common predictors, with a notable absence of social variables. Frequently used ML models included support vector machines and tree-based methods. ML-enhanced models typically outperformed conventional triage algorithms, and we found a wide range of methods used to establish ground truth labels. Conclusions: This scoping review observed heterogeneity in dataset size, predictors, clinical setting (triage process), and reported performance metrics. Standard structured predictors, including age, sex, and comorbidities, across articles suggest the importance of these inputs; however, there was a notable absence of other potentially useful data, including medications, social variables, and health system exposure. Ground truth labeling practices should be reported in a standard fashion as the true model performance hinges on these labels. This review calls for future work to form a standardized framework, thereby supporting consistent reporting and performance comparisons across ML-enhanced prehospital triage systems.
- 11 September 2024
- Published by JMIR Publications Inc. in Interactive Journal of Medical Research
- Vol. 13 (1) , e56729
Open AccessImproving Triage Accuracy in Prehospital Emergency Telemedicine: Scoping Review of Machine Learning–Enhanced Approaches
Background: Prehospital telemedicine triage systems combined with machine learning (ML) methods have the potential to improve triage accuracy and safely redirect low-acuity patients from attending the emergency department. However, research in prehospital settings is limited but needed; emergency department overcrowding and adverse patient outcomes are increasingly common. Objective: In this scoping review, we sought to characterize the existing methods for ML-enhanced telemedicine emergency triage. In order to support future research, we aimed to delineate what data sources, predictors, labels, ML models, and performance metrics were used, and in which telemedicine triage systems these methods were applied. Methods: A scoping review was conducted, querying multiple databases (MEDLINE, PubMed, Scopus, and IEEE Xplore) through February 24, 2023, to identify potential ML-enhanced methods, and for those eligible, relevant study characteristics were extracted, including prehospital triage setting, types of predictors, ground truth labeling method, ML models used, and performance metrics. Inclusion criteria were restricted to the triage of emergency telemedicine services using ML methods on an undifferentiated (disease nonspecific) population. Only primary research studies in English were considered. Furthermore, only those studies using data collected remotely (as opposed to derived from physical assessments) were included. In order to limit bias, we exclusively included articles identified through our predefined search criteria and had 3 researchers (DR, JS, and KS) independently screen the resulting studies. We conducted a narrative synthesis of findings to establish a knowledge base in this domain and identify potential gaps to be addressed in forthcoming ML-enhanced methods. Results: A total of 165 unique records were screened for eligibility and 15 were included in the review. Most studies applied ML methods during emergency medical dispatch (7/15, 47%) or used chatbot applications (5/15, 33%). Patient demographics and health status variables were the most common predictors, with a notable absence of social variables. Frequently used ML models included support vector machines and tree-based methods. ML-enhanced models typically outperformed conventional triage algorithms, and we found a wide range of methods used to establish ground truth labels. Conclusions: This scoping review observed heterogeneity in dataset size, predictors, clinical setting (triage process), and reported performance metrics. Standard structured predictors, including age, sex, and comorbidities, across articles suggest the importance of these inputs; however, there was a notable absence of other potentially useful data, including medications, social variables, and health system exposure. Ground truth labeling practices should be reported in a standard fashion as the true model performance hinges on these labels. This review calls for future work to form a standardized framework, thereby supporting consistent reporting and performance comparisons across ML-enhanced prehospital triage systems.
- Open Access
- 1 February 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Estimating Patient and Family Costs and CO2 Emissions for Telehealth and In-Person Health Care Appointments in British Columbia, Canada: Geospatial Mixed Methods Study (Preprint)
BACKGROUND Patients inevitably incur some cost for accessing health care, even in universal systems such as Canada. The COVID-19 pandemic dramatically shifted health care delivery from in-person to telehealth services, also shifting the proportion of costs offset by patients and their families by reducing the need to travel to in-person appointments. OBJECTIVE This study aimed to develop a method for estimating the costs patients and their families incur and CO2 emissions attributed to travel needed for emergency department (ED) visits, hospitalizations, and physician appointments. METHODS We present a method to evaluate the costs associated with in-person and telehealth care appointments from the perspective of patients, their families, and the environment. We used ED locations, road distances, and duration of appointment to account for costs paid by patients (ie, lost productivity, informal caregiving, and out-of-pocket expenses) attributed to travel to receive medical care. Costs to the environment were evaluated by calculating the amount of CO2 emitted per medical visit. Using our costs calculated per visit, we apply our method to calculate total patient costs for a simulated population over 1 year. RESULTS Our method estimates that patients in British Columbia pay up to $300 (2023 CAD, CAD $1=US $0.86) on average to attend an in-person ED visit, depending on where they live; $166 may be attributed to lost productivity, $83 to informal caregiving, and $50 to out-of-pocket expenses. These estimates are higher than most observed cost estimates. In addition, avoiding in-person care diverts up to 13 kg of CO2 per medical visit, depending on the distance and frequency of travel to appointments. This translates to up to $0.70 in carbon costs per visit, or cumulatively $44,120 per year in British Columbia, conventionally not included in patient cost estimates. CONCLUSIONS We present a novel method for estimating patient-incurred costs and CO2 emissions from accessing health care and apply it to estimate that every year, patients in British Columbia pay upwards of 30 million dollars to access health care services, primarily for medical travel. Our method adds to the economic evaluation literature by providing a more comprehensive and context-modifiable calculation of patient costs that will allow for more informed decision-making regarding health care services.
- 1 February 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessEstimating Patient and Family Costs and CO2 Emissions for Telehealth and In-Person Health Care Appointments in British Columbia, Canada: Geospatial Mixed Methods Study (Preprint)
BACKGROUND Patients inevitably incur some cost for accessing health care, even in universal systems such as Canada. The COVID-19 pandemic dramatically shifted health care delivery from in-person to telehealth services, also shifting the proportion of costs offset by patients and their families by reducing the need to travel to in-person appointments. OBJECTIVE This study aimed to develop a method for estimating the costs patients and their families incur and CO2 emissions attributed to travel needed for emergency department (ED) visits, hospitalizations, and physician appointments. METHODS We present a method to evaluate the costs associated with in-person and telehealth care appointments from the perspective of patients, their families, and the environment. We used ED locations, road distances, and duration of appointment to account for costs paid by patients (ie, lost productivity, informal caregiving, and out-of-pocket expenses) attributed to travel to receive medical care. Costs to the environment were evaluated by calculating the amount of CO2 emitted per medical visit. Using our costs calculated per visit, we apply our method to calculate total patient costs for a simulated population over 1 year. RESULTS Our method estimates that patients in British Columbia pay up to $300 (2023 CAD, CAD $1=US $0.86) on average to attend an in-person ED visit, depending on where they live; $166 may be attributed to lost productivity, $83 to informal caregiving, and $50 to out-of-pocket expenses. These estimates are higher than most observed cost estimates. In addition, avoiding in-person care diverts up to 13 kg of CO2 per medical visit, depending on the distance and frequency of travel to appointments. This translates to up to $0.70 in carbon costs per visit, or cumulatively $44,120 per year in British Columbia, conventionally not included in patient cost estimates. CONCLUSIONS We present a novel method for estimating patient-incurred costs and CO2 emissions from accessing health care and apply it to estimate that every year, patients in British Columbia pay upwards of 30 million dollars to access health care services, primarily for medical travel. Our method adds to the economic evaluation literature by providing a more comprehensive and context-modifiable calculation of patient costs that will allow for more informed decision-making regarding health care services.
- Open Access
- 24 January 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Improving Triage Accuracy in Prehospital Emergency Telemedicine: Scoping Review of Machine Learning–Enhanced Approaches (Preprint)
BACKGROUND Prehospital telemedicine triage systems combined with machine learning (ML) methods have the potential to improve triage accuracy and safely redirect low-acuity patients from attending the emergency department. However, research in prehospital settings is limited but needed; emergency department overcrowding and adverse patient outcomes are increasingly common. OBJECTIVE In this scoping review, we sought to characterize the existing methods for ML-enhanced telemedicine emergency triage. In order to support future research, we aimed to delineate what data sources, predictors, labels, ML models, and performance metrics were used, and in which telemedicine triage systems these methods were applied. METHODS A scoping review was conducted, querying multiple databases (MEDLINE, PubMed, Scopus, and IEEE Xplore) through February 24, 2023, to identify potential ML-enhanced methods, and for those eligible, relevant study characteristics were extracted, including prehospital triage setting, types of predictors, ground truth labeling method, ML models used, and performance metrics. Inclusion criteria were restricted to the triage of emergency telemedicine services using ML methods on an undifferentiated (disease nonspecific) population. Only primary research studies in English were considered. Furthermore, only those studies using data collected remotely (as opposed to derived from physical assessments) were included. In order to limit bias, we exclusively included articles identified through our predefined search criteria and had 3 researchers (DR, JS, and KS) independently screen the resulting studies. We conducted a narrative synthesis of findings to establish a knowledge base in this domain and identify potential gaps to be addressed in forthcoming ML-enhanced methods. RESULTS A total of 165 unique records were screened for eligibility and 15 were included in the review. Most studies applied ML methods during emergency medical dispatch (7/15, 47%) or used chatbot applications (5/15, 33%). Patient demographics and health status variables were the most common predictors, with a notable absence of social variables. Frequently used ML models included support vector machines and tree-based methods. ML-enhanced models typically outperformed conventional triage algorithms, and we found a wide range of methods used to establish ground truth labels. CONCLUSIONS This scoping review observed heterogeneity in dataset size, predictors, clinical setting (triage process), and reported performance metrics. Standard structured predictors, including age, sex, and comorbidities, across articles suggest the importance of these inputs; however, there was a notable absence of other potentially useful data, including medications, social variables, and health system exposure. Ground truth labeling practices should be reported in a standard fashion as the true model performance hinges on these labels. This review calls for future work to form a standardized framework, thereby supporting consistent reporting and performance comparisons across ML-enhanced prehospital triage systems.
- 24 January 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessImproving Triage Accuracy in Prehospital Emergency Telemedicine: Scoping Review of Machine Learning–Enhanced Approaches (Preprint)
BACKGROUND Prehospital telemedicine triage systems combined with machine learning (ML) methods have the potential to improve triage accuracy and safely redirect low-acuity patients from attending the emergency department. However, research in prehospital settings is limited but needed; emergency department overcrowding and adverse patient outcomes are increasingly common. OBJECTIVE In this scoping review, we sought to characterize the existing methods for ML-enhanced telemedicine emergency triage. In order to support future research, we aimed to delineate what data sources, predictors, labels, ML models, and performance metrics were used, and in which telemedicine triage systems these methods were applied. METHODS A scoping review was conducted, querying multiple databases (MEDLINE, PubMed, Scopus, and IEEE Xplore) through February 24, 2023, to identify potential ML-enhanced methods, and for those eligible, relevant study characteristics were extracted, including prehospital triage setting, types of predictors, ground truth labeling method, ML models used, and performance metrics. Inclusion criteria were restricted to the triage of emergency telemedicine services using ML methods on an undifferentiated (disease nonspecific) population. Only primary research studies in English were considered. Furthermore, only those studies using data collected remotely (as opposed to derived from physical assessments) were included. In order to limit bias, we exclusively included articles identified through our predefined search criteria and had 3 researchers (DR, JS, and KS) independently screen the resulting studies. We conducted a narrative synthesis of findings to establish a knowledge base in this domain and identify potential gaps to be addressed in forthcoming ML-enhanced methods. RESULTS A total of 165 unique records were screened for eligibility and 15 were included in the review. Most studies applied ML methods during emergency medical dispatch (7/15, 47%) or used chatbot applications (5/15, 33%). Patient demographics and health status variables were the most common predictors, with a notable absence of social variables. Frequently used ML models included support vector machines and tree-based methods. ML-enhanced models typically outperformed conventional triage algorithms, and we found a wide range of methods used to establish ground truth labels. CONCLUSIONS This scoping review observed heterogeneity in dataset size, predictors, clinical setting (triage process), and reported performance metrics. Standard structured predictors, including age, sex, and comorbidities, across articles suggest the importance of these inputs; however, there was a notable absence of other potentially useful data, including medications, social variables, and health system exposure. Ground truth labeling practices should be reported in a standard fashion as the true model performance hinges on these labels. This review calls for future work to form a standardized framework, thereby supporting consistent reporting and performance comparisons across ML-enhanced prehospital triage systems.
- Open Access
- 22 December 2023
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 25, e45451
Real-Time Virtual Support as an Emergency Department Strategy for Rural, Remote, and Indigenous Communities in British Columbia: Descriptive Case Study
Background: British Columbia has over 200 rural, remote, and Indigenous communities that have limited health care resources due to physician isolation, sparsity in clinical resources, the lack of collegial support, and provider burnout. Real-time virtual support (RTVS) peer-to-peer pathways provide support to patients and providers. Amid the COVID-19 pandemic exacerbating existing health care disparities and equitable access to timely care, RTVS presents a portable and additional opportunity to be deployed in a hospital or patient home setting in rural communities. We highlight the story of the Rural Urgent Doctor in-aid (RUDi) pathway within RTVS that successfully supported the Dawson Creek District Hospital (DCDH) emergency department (ED) in 2021. Objective: This study aims to describe the rapid implementation process and identify facilitators and barriers to successful implementation. Methods: This case study is grounded in the Quadruple Aim and Social Accountability frameworks for health systems learning. The entire study period was approximately 6 months. After 1 week of implementation, we interviewed RUDi physicians, DCDH staff, health authority leadership, and RTVS staff to gather their experiences. Content analysis was used to identify themes that emerged from the interviews. Results: RUDi physicians covered 39 overnight shifts and were the most responsible providers (MRPs) for 245 patients who presented to the DCDH ED. A total of 17 interviews with key informants revealed important themes related to leadership and relationships as facilitators of the coverage’s success, the experience of remote physician support, providing a “safety net,” finding new ways of interprofessional collaboration, and the need for extensive IT support throughout. Quality improvement findings identified barriers and demonstrated tangible recommendations for how this model of support can be improved in future cases. Conclusions: By acting as the MRP during overnight ED shifts, RUDi prevented the closure of the DCDH ED and the diversion of patients to another rural hospital. Rapid codevelopment and implementation of digital health solutions can be leveraged with existing partnerships and mutual trust between RTVS and rural EDs to ease the pressures of a physician shortage, particularly during COVID-19. By establishing new and modified clinical workflows, RTVS provides a safety net for rural patients and providers challenged by burnout. This case study provides learnings to be implemented to serve future rural, remote, and Indigenous communities in crisis.
- 22 December 2023
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 25, e45451
Open AccessReal-Time Virtual Support as an Emergency Department Strategy for Rural, Remote, and Indigenous Communities in British Columbia: Descriptive Case Study
Background: British Columbia has over 200 rural, remote, and Indigenous communities that have limited health care resources due to physician isolation, sparsity in clinical resources, the lack of collegial support, and provider burnout. Real-time virtual support (RTVS) peer-to-peer pathways provide support to patients and providers. Amid the COVID-19 pandemic exacerbating existing health care disparities and equitable access to timely care, RTVS presents a portable and additional opportunity to be deployed in a hospital or patient home setting in rural communities. We highlight the story of the Rural Urgent Doctor in-aid (RUDi) pathway within RTVS that successfully supported the Dawson Creek District Hospital (DCDH) emergency department (ED) in 2021. Objective: This study aims to describe the rapid implementation process and identify facilitators and barriers to successful implementation. Methods: This case study is grounded in the Quadruple Aim and Social Accountability frameworks for health systems learning. The entire study period was approximately 6 months. After 1 week of implementation, we interviewed RUDi physicians, DCDH staff, health authority leadership, and RTVS staff to gather their experiences. Content analysis was used to identify themes that emerged from the interviews. Results: RUDi physicians covered 39 overnight shifts and were the most responsible providers (MRPs) for 245 patients who presented to the DCDH ED. A total of 17 interviews with key informants revealed important themes related to leadership and relationships as facilitators of the coverage’s success, the experience of remote physician support, providing a “safety net,” finding new ways of interprofessional collaboration, and the need for extensive IT support throughout. Quality improvement findings identified barriers and demonstrated tangible recommendations for how this model of support can be improved in future cases. Conclusions: By acting as the MRP during overnight ED shifts, RUDi prevented the closure of the DCDH ED and the diversion of patients to another rural hospital. Rapid codevelopment and implementation of digital health solutions can be leveraged with existing partnerships and mutual trust between RTVS and rural EDs to ease the pressures of a physician shortage, particularly during COVID-19. By establishing new and modified clinical workflows, RTVS provides a safety net for rural patients and providers challenged by burnout. This case study provides learnings to be implemented to serve future rural, remote, and Indigenous communities in crisis.