KSKurtis Stewart

Kurtis Stewart

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Kurtis Stewart has a total of 122 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Patient concordance following an urgent virtual physician consultation in a provincial triage program in British Columbia, Canada (Preprint)

    by Kurtis Stewart, Zenith A Zyn, Swati Kanwal, Devon Currie, Daniel Raff, Ross Duncan, Lindsay Hedden, Brandon Wong, Sandra Sundhu, Frank X Scheuermeyer, et al.

    BACKGROUND Virtual nurse-physician collaborations in pre-hospital triage program such as HealthLink BC’s 8-1-1 service can reduce unnecessary emergency department (ED) by referring patients to the appropriate level of follow-up care, ranging from “Go to ED Now” to “Home Treatment”. However, limited evidence exists on the level to which patients follow through with the advice provided. OBJECTIVE We sought to describe the patient-level and geographic factors associated with caller concordance to 8-1-1 physician advice and develop predictive machine learning (ML) models of concordance. METHODS All calls to the 8-1-1 virtual physician service made between April 6, 2020 and March 31, 2023 with a follow-up care recommendation were eligible. Caller records were linked to provincial health administrative databases to determine demographic and pre-/post-call health service use information and geographic information based on health or census area. Caller concordance was calculated by physician discharge disposition. Descriptive analyses summarized concordance rates by demographic characteristics and health regions. ML models, including support vector machines, neural networks, and extreme gradient boosting (XGBoost), were developed using over 380 preprocessed features to predict concordance and evaluated using F1-scores. RESULTS 113,090 calls were included in the analysis. Concordance rates varied by physician disposition: they were highest for “Go to ED Now” calls (61.3%). When stratified, concordance rates were most strongly associated with health region and rurality. The XGBoost model had the strongest predictive performance for the “Go to ED Now” disposition (F1-score=0.78), while other models and other dispositions had F1-scores<=0.63. Health region, rurality, sex, and primary care attachment were consistently the strongest features influencing model predictions. In a sensitivity analysis expanding the concordance definitions, XGBoost models’ F1-scores increased up to 0.90. CONCLUSIONS Caller concordance with virtual physician advice was highest for the most urgent care recommendation, but varied greatly by health region, indicating significant disparities for rural patients trying to access immediate ED care. XGBoost models had strong predictive performance for more urgent care recommendations, especially when considering concordance over longer timeframes. ML models could be leveraged to support patients at risk of discordance and subsequent potentially negative outcomes by personalizing virtual triage interventions.

  • Open Access

    Healthcare Providers’ Experiences Accessing Real-Time Virtual Support: Informing More Equitable and Inclusive Healthcare Access in British Columbia’s Rural, Remote, First Nations, and Other Indigenous Peoples and Communities

    by Hollis Owens, Nazia Nadir Shah, Michelle Lin, Rochelle Chauhan, Joan Assali, Amrit Bhullar, Kurtis Stewart, Kendall Ho, Anne Lesack, Erika Pritchard, et al.

    Healthcare Providers (HCPs) serving Rural, Remote, First Nations, and other Indigenous (RRFNI) communities face unique challenges in delivering longitudinal care due to geographic isolation. The Real-Time Virtual Support Services (RTVS) network aims to improve equitable access to healthcare and provide collegial support for HCPs in RRFNI communities across British Columbia. The objective of this study was to understand HCPs’ experiences with RTVS and identify improvement areas. Data were collected through semi-structured interviews with HCPs that were recorded, transcribed, and openly coded. Twenty HCPs using RTVS were interviewed during 2022-2023. The constant comparative method was used to develop themes. Themes focused on RTVS’s benefits and outcomes including increased clinical confidence, reduced provider anxiety, respectful and collegial support, reduced administrative burden, and recruitment and retention support. Challenges included occasional service disruptions and limited Wi-Fi availability. These findings provide in-depth and contextualized feedback informing the development of RTVS.

  • VIVA: Virtual Healthcare Interactions Using Visual Analytics, With Controllability Through Configuration

    by Jürgen Bernard, Mara Solen, Helen Novak Lauscher, Kurtis Stewart, Kendall Ho, Tamara Munzner

    At the beginning of the COVID-19 pandemic, HealthLink BC (HLBC) rapidly integrated physicians into the triage process of their virtual healthcare service to improve patient outcomes and satisfaction with this service and preserve health care system capacity. We present the design and implementation of a visual analytics tool, VIVA (Virtual healthcare Interactions using Visual Analytics), to support HLBC in analysing various forms of usage data from the service. We abstract HLBC’s data and data analysis tasks, which we use to inform our design of VIVA. We also present the interactive workflow abstraction of Scan, Act, Adapt. We validate VIVA’s design through three case studies with stakeholder domain experts. We also propose the Controllability Through Configuration model to conduct and analyze design studies, and discuss architectural evolution of VIVA through that lens. It articulates configuration, both that specified by a developer or technical power user and that constructed automatically through log data from previous interactive sessions, as a bridge between the rigidity of hardwired programming and the time-consuming implementation of full end-user interactivity.


  • Open Access

    VIVA: Virtual Healthcare Interactions Using Visual Analytics, With Controllability Through Configuration

    by Jürgen Bernard, Mara Solen, Helen Novak Lauscher, Kurtis Stewart, Kendall Ho, Tamara Munzner

    At the beginning of the COVID-19 pandemic, HealthLink BC (HLBC) rapidly integrated physicians into the triage process of their virtual healthcare service to improve patient outcomes and satisfaction with this service and preserve health care system capacity. We present the design and implementation of a visual analytics tool, VIVA (Virtual healthcare Interactions using Visual Analytics), to support HLBC in analysing various forms of usage data from the service. We abstract HLBC's data and data analysis tasks, which we use to inform our design of VIVA. We also present the interactive workflow abstraction of Scan, Act, Adapt. We validate VIVA's design through three case studies with stakeholder domain experts. We also propose the Controllability Through Configuration model to conduct and analyze design studies, and discuss architectural evolution of VIVA through that lens. It articulates configuration, both that specified by a developer or technical power user and that constructed automatically through log data from previous interactive sessions, as a bridge between the rigidity of hardwired programming and the time-consuming implementation of full end-user interactivity. Availability: Supplemental materials at https://osf.io/wv38n

  • Open Access

    Physicians’ experiences delivering provincial real-time virtual support services: A qualitative interview study

    by Michelle Yang, Kurtis Stewart, Amrit Bhullar, Joanne Chi, Elsie Jiaxi Wang, Ivjot Samra, Kendall Ho, Joan Assali, Mina Han, Helen Novak Lauscher

    Rural, remote, and Indigenous communities in British Columbia (BC) tend to have lower access to healthcare providers and poorer health outcomes—an inequality that the COVID-19 pandemic has exacerbated. In response, Real-Time Virtual Support (RTVS) pathways were developed to advance equitable access to care for patients and provide peer support to physicians working in underserved communities. This study aimed to describe the perspectives of Virtual Physicians (VPs) who delivered the RTVS services. Forty-five RTVS VPs engaged in 30-minute semi-structured interviews about their experiences and perspectives delivering RTVS. Three themes emerged: (1) RTVS’s contributions to VPs’ personal and professional development; (2) impacts on communities; and (3) considerations for the availability and expansion. VPs identified incremental expansion and attaining funding stability as critical next steps for virtual healthcare in BC. This evidence informed RTVS program evaluation and may provide learnings relevant to other jurisdictions.


  • Open Access

    Estimating Patient and Family Costs and CO2 Emissions for Telehealth and In-Person Health Care Appointments in British Columbia, Canada: Geospatial Mixed Methods Study

    by Graham Mainer-Pearson, Kurtis Stewart, Kim Williams, John Pawlovich, Scott Graham, Linda Riches, Sonya Cressman, Kendall Ho

    Background: Patients inevitably incur some cost for accessing health care, even in universal systems such as Canada. The COVID-19 pandemic dramatically shifted health care delivery from in-person to telehealth services, also shifting the proportion of costs offset by patients and their families by reducing the need to travel to in-person appointments. Objective: This study aimed to develop a method for estimating the costs patients and their families incur and CO2 emissions attributed to travel needed for emergency department (ED) visits, hospitalizations, and physician appointments. Methods: We present a method to evaluate the costs associated with in-person and telehealth care appointments from the perspective of patients, their families, and the environment. We used ED locations, road distances, and duration of appointment to account for costs paid by patients (ie, lost productivity, informal caregiving, and out-of-pocket expenses) attributed to travel to receive medical care. Costs to the environment were evaluated by calculating the amount of CO2 emitted per medical visit. Using our costs calculated per visit, we apply our method to calculate total patient costs for a simulated population over 1 year. Results: Our method estimates that patients in British Columbia pay up to $300 (2023 CAD, CAD $1=US $0.86) on average to attend an in-person ED visit, depending on where they live; $166 may be attributed to lost productivity, $83 to informal caregiving, and $50 to out-of-pocket expenses. These estimates are higher than most observed cost estimates. In addition, avoiding in-person care diverts up to 13 kg of CO2 per medical visit, depending on the distance and frequency of travel to appointments. This translates to up to $0.70 in carbon costs per visit, or cumulatively $44,120 per year in British Columbia, conventionally not included in patient cost estimates. Conclusions: We present a novel method for estimating patient-incurred costs and CO2 emissions from accessing health care and apply it to estimate that every year, patients in British Columbia pay upwards of 30 million dollars to access health care services, primarily for medical travel. Our method adds to the economic evaluation literature by providing a more comprehensive and context-modifiable calculation of patient costs that will allow for more informed decision-making regarding health care services.


  • Open Access

    Improving Triage Accuracy in Prehospital Emergency Telemedicine: Scoping Review of Machine Learning–Enhanced Approaches

    by Daniel Raff, Kurtis Stewart, Michelle Christie Yang, Jessie Shang, Sonya Cressman, Roger Tam, Jessica Wong, Martin C Tammemägi, Kendall Ho

    Background: Prehospital telemedicine triage systems combined with machine learning (ML) methods have the potential to improve triage accuracy and safely redirect low-acuity patients from attending the emergency department. However, research in prehospital settings is limited but needed; emergency department overcrowding and adverse patient outcomes are increasingly common. Objective: In this scoping review, we sought to characterize the existing methods for ML-enhanced telemedicine emergency triage. In order to support future research, we aimed to delineate what data sources, predictors, labels, ML models, and performance metrics were used, and in which telemedicine triage systems these methods were applied. Methods: A scoping review was conducted, querying multiple databases (MEDLINE, PubMed, Scopus, and IEEE Xplore) through February 24, 2023, to identify potential ML-enhanced methods, and for those eligible, relevant study characteristics were extracted, including prehospital triage setting, types of predictors, ground truth labeling method, ML models used, and performance metrics. Inclusion criteria were restricted to the triage of emergency telemedicine services using ML methods on an undifferentiated (disease nonspecific) population. Only primary research studies in English were considered. Furthermore, only those studies using data collected remotely (as opposed to derived from physical assessments) were included. In order to limit bias, we exclusively included articles identified through our predefined search criteria and had 3 researchers (DR, JS, and KS) independently screen the resulting studies. We conducted a narrative synthesis of findings to establish a knowledge base in this domain and identify potential gaps to be addressed in forthcoming ML-enhanced methods. Results: A total of 165 unique records were screened for eligibility and 15 were included in the review. Most studies applied ML methods during emergency medical dispatch (7/15, 47%) or used chatbot applications (5/15, 33%). Patient demographics and health status variables were the most common predictors, with a notable absence of social variables. Frequently used ML models included support vector machines and tree-based methods. ML-enhanced models typically outperformed conventional triage algorithms, and we found a wide range of methods used to establish ground truth labels. Conclusions: This scoping review observed heterogeneity in dataset size, predictors, clinical setting (triage process), and reported performance metrics. Standard structured predictors, including age, sex, and comorbidities, across articles suggest the importance of these inputs; however, there was a notable absence of other potentially useful data, including medications, social variables, and health system exposure. Ground truth labeling practices should be reported in a standard fashion as the true model performance hinges on these labels. This review calls for future work to form a standardized framework, thereby supporting consistent reporting and performance comparisons across ML-enhanced prehospital triage systems.


  • Open Access

    Estimating Patient and Family Costs and CO2 Emissions for Telehealth and In-Person Health Care Appointments in British Columbia, Canada: Geospatial Mixed Methods Study (Preprint)

    by Graham Mainer-Pearson, Kurtis Stewart, Kim Williams, John Pawlovich, Scott Graham, Linda Riches, Sonya Cressman, Kendall Ho

    BACKGROUND Patients inevitably incur some cost for accessing health care, even in universal systems such as Canada. The COVID-19 pandemic dramatically shifted health care delivery from in-person to telehealth services, also shifting the proportion of costs offset by patients and their families by reducing the need to travel to in-person appointments. OBJECTIVE This study aimed to develop a method for estimating the costs patients and their families incur and CO2 emissions attributed to travel needed for emergency department (ED) visits, hospitalizations, and physician appointments. METHODS We present a method to evaluate the costs associated with in-person and telehealth care appointments from the perspective of patients, their families, and the environment. We used ED locations, road distances, and duration of appointment to account for costs paid by patients (ie, lost productivity, informal caregiving, and out-of-pocket expenses) attributed to travel to receive medical care. Costs to the environment were evaluated by calculating the amount of CO2 emitted per medical visit. Using our costs calculated per visit, we apply our method to calculate total patient costs for a simulated population over 1 year. RESULTS Our method estimates that patients in British Columbia pay up to $300 (2023 CAD, CAD $1=US $0.86) on average to attend an in-person ED visit, depending on where they live; $166 may be attributed to lost productivity, $83 to informal caregiving, and $50 to out-of-pocket expenses. These estimates are higher than most observed cost estimates. In addition, avoiding in-person care diverts up to 13 kg of CO2 per medical visit, depending on the distance and frequency of travel to appointments. This translates to up to $0.70 in carbon costs per visit, or cumulatively $44,120 per year in British Columbia, conventionally not included in patient cost estimates. CONCLUSIONS We present a novel method for estimating patient-incurred costs and CO2 emissions from accessing health care and apply it to estimate that every year, patients in British Columbia pay upwards of 30 million dollars to access health care services, primarily for medical travel. Our method adds to the economic evaluation literature by providing a more comprehensive and context-modifiable calculation of patient costs that will allow for more informed decision-making regarding health care services.


  • Open Access

    Improving Triage Accuracy in Prehospital Emergency Telemedicine: Scoping Review of Machine Learning–Enhanced Approaches (Preprint)

    by Daniel Raff, Kurtis Stewart, Michelle Christie Yang, Jessie Shang, Sonya Cressman, Roger Tam, Jessica Wong, Martin C Tammemägi, Kendall Ho

    BACKGROUND Prehospital telemedicine triage systems combined with machine learning (ML) methods have the potential to improve triage accuracy and safely redirect low-acuity patients from attending the emergency department. However, research in prehospital settings is limited but needed; emergency department overcrowding and adverse patient outcomes are increasingly common. OBJECTIVE In this scoping review, we sought to characterize the existing methods for ML-enhanced telemedicine emergency triage. In order to support future research, we aimed to delineate what data sources, predictors, labels, ML models, and performance metrics were used, and in which telemedicine triage systems these methods were applied. METHODS A scoping review was conducted, querying multiple databases (MEDLINE, PubMed, Scopus, and IEEE Xplore) through February 24, 2023, to identify potential ML-enhanced methods, and for those eligible, relevant study characteristics were extracted, including prehospital triage setting, types of predictors, ground truth labeling method, ML models used, and performance metrics. Inclusion criteria were restricted to the triage of emergency telemedicine services using ML methods on an undifferentiated (disease nonspecific) population. Only primary research studies in English were considered. Furthermore, only those studies using data collected remotely (as opposed to derived from physical assessments) were included. In order to limit bias, we exclusively included articles identified through our predefined search criteria and had 3 researchers (DR, JS, and KS) independently screen the resulting studies. We conducted a narrative synthesis of findings to establish a knowledge base in this domain and identify potential gaps to be addressed in forthcoming ML-enhanced methods. RESULTS A total of 165 unique records were screened for eligibility and 15 were included in the review. Most studies applied ML methods during emergency medical dispatch (7/15, 47%) or used chatbot applications (5/15, 33%). Patient demographics and health status variables were the most common predictors, with a notable absence of social variables. Frequently used ML models included support vector machines and tree-based methods. ML-enhanced models typically outperformed conventional triage algorithms, and we found a wide range of methods used to establish ground truth labels. CONCLUSIONS This scoping review observed heterogeneity in dataset size, predictors, clinical setting (triage process), and reported performance metrics. Standard structured predictors, including age, sex, and comorbidities, across articles suggest the importance of these inputs; however, there was a notable absence of other potentially useful data, including medications, social variables, and health system exposure. Ground truth labeling practices should be reported in a standard fashion as the true model performance hinges on these labels. This review calls for future work to form a standardized framework, thereby supporting consistent reporting and performance comparisons across ML-enhanced prehospital triage systems.

  • Open Access

    Real-Time Virtual Support as an Emergency Department Strategy for Rural, Remote, and Indigenous Communities in British Columbia: Descriptive Case Study

    by Helen Novak Lauscher, Brydon Blacklaws, Erika Pritchard, Elsie Jiaxi Wang, Kurtis Stewart, Jeff Beselt, Kendall Ho, John Pawlovich

    Background: British Columbia has over 200 rural, remote, and Indigenous communities that have limited health care resources due to physician isolation, sparsity in clinical resources, the lack of collegial support, and provider burnout. Real-time virtual support (RTVS) peer-to-peer pathways provide support to patients and providers. Amid the COVID-19 pandemic exacerbating existing health care disparities and equitable access to timely care, RTVS presents a portable and additional opportunity to be deployed in a hospital or patient home setting in rural communities. We highlight the story of the Rural Urgent Doctor in-aid (RUDi) pathway within RTVS that successfully supported the Dawson Creek District Hospital (DCDH) emergency department (ED) in 2021. Objective: This study aims to describe the rapid implementation process and identify facilitators and barriers to successful implementation. Methods: This case study is grounded in the Quadruple Aim and Social Accountability frameworks for health systems learning. The entire study period was approximately 6 months. After 1 week of implementation, we interviewed RUDi physicians, DCDH staff, health authority leadership, and RTVS staff to gather their experiences. Content analysis was used to identify themes that emerged from the interviews. Results: RUDi physicians covered 39 overnight shifts and were the most responsible providers (MRPs) for 245 patients who presented to the DCDH ED. A total of 17 interviews with key informants revealed important themes related to leadership and relationships as facilitators of the coverage’s success, the experience of remote physician support, providing a “safety net,” finding new ways of interprofessional collaboration, and the need for extensive IT support throughout. Quality improvement findings identified barriers and demonstrated tangible recommendations for how this model of support can be improved in future cases. Conclusions: By acting as the MRP during overnight ED shifts, RUDi prevented the closure of the DCDH ED and the diversion of patients to another rural hospital. Rapid codevelopment and implementation of digital health solutions can be leveraged with existing partnerships and mutual trust between RTVS and rural EDs to ease the pressures of a physician shortage, particularly during COVID-19. By establishing new and modified clinical workflows, RTVS provides a safety net for rural patients and providers challenged by burnout. This case study provides learnings to be implemented to serve future rural, remote, and Indigenous communities in crisis.