Alexander Singer

Background: Health care systems generate vast amounts of unstructured text, such as clinical notes, which capture nuanced patient experiences, clinical reasoning, and subtle indicators of health status. While health system research has traditionally relied upon structured data, natural language processing (NLP) enables the extraction of this rich textual information. Leveraging NLP could improve the identification and characterization of underreported adverse drug events (ADEs). Objective: The primary objective of this study was to train and evaluate multiple NLP models, including both previously published architectures and a novel model, for the identification of ADEs from clinical notes. Methods: Electronic medical records from the Manitoba Primary Care Research Network (MaPCReN) were used in this study. Clinical notes were annotated to indicate the presence of a possible ADE, the associated words or phrases, and the corresponding drug. A subselection algorithm was applied to ensure sufficient representation of notes containing ADEs for training a robust classifier. The cohort was restricted to patients aged 55 years and older and was annotated in 2 waves: Wave 1 comprised primary care encounter notes selected for a temporally linked emergency department (ED) visit, enriching it for acute presentations, while Wave 2 relaxed this requirement, and its acuity composition was uncharacterized. The annotated data were split into training and test sets. NLP models—including BioBERT, BlueBERT, a large language model (LLM) classifier, and an LLM embeddings–based classifier—were trained on both original clinical notes and notes reformatted into Subjective-Objective-Assessment-Plan (SOAP) structure. To approximate a clinician-inspired reasoning workflow, Mistral-7b-Instruct was used to extract presenting symptoms and generate a ranked list of potential etiologies. Model performance was evaluated using precision, recall, and F ₁ -score. Results: Of the 1085 annotated encounter notes, 355 included ADEs. Across 9 modeling approaches evaluated over 5 random seeds, models trained on SOAP-rewritten notes generally outperformed those trained on original notes. The SOAP Rewrite+ LLM Embeddings Classifier (GritLM-SOAP) achieved the highest mean F ₁ -score (72.53%; 95% CI 62.40%‐81.82%), while BioBERT-SOAP weighted achieved the highest mean recall (76.34%; 95% CI 62.69%‐89.66%). End-to-end span-level extraction (named entity recognition+relation extraction) on ADE-positive test notes achieved a mean relaxed F ₁ -score of 0.455, with relation extraction identified as the bottleneck. Conclusions: Effective detection of ADEs in clinical notes may benefit from NLP models that approximate the clinical reasoning of health care providers. While the SOAP Rewrite+ LLM Embeddings Classifier demonstrated a reasonable balance of precision and recall, there is room for improvement as models evolve.

To validate a primary care electronic medical record (EMR) case definition for mood and anxiety disorders (including depression, anxiety, and bipolar disorder) and schizophrenia that can be used to estimate prevalence and co-occurrence. Retrospective cross-sectional study. Canada. De-identified EMR data was used from 1574 primary care providers participating in the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) from 1,692,987 patients who had 1 or more visits with a primary care provider. The reference set included 2488 patients, with 434 positive and 2054 negative for 1 or more mental health conditions of interest. A second reference set for schizophrenia represented 760 patients (30 positive and 730 negative). The agreement of 29 case definitions was assessed against a reference set by reporting sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy. Prevalence was estimated and co-occurrence was assessed in the CPCSSN dataset (N=1,692,987). The strongest definition for mood disorders captured anxiety, depression, and bipolar disorder with a sensitivity of 80.7%, specificity of 88.7%, PPV of 59.9%, and NPV of 95.7%; and an estimated prevalence of 21.8% (95% CI 21.7 to 21.9). The inclusion of psychosis did not improve agreement (sensitivity 95.2%, specificity 80.7%, PPV 51.0%, NPV 98.8%), but schizophrenia alone had high agreement (sensitivity 93.3%, specificity 100%, PPV 100%, NPV 99.9%). High co-occurrence of anxiety, depression, and bipolar disorder was found. Algorithms validated to capture these conditions together produced stronger agreement compared with individual definitions. Schizophrenia was less likely to co-occur with other mental health conditions and produced higher agreement when validated separately. Application of validated algorithms to capture mental health conditions can inform disease surveillance and health system planning.

Objectives: We assessed trends in primary care paediatric blood pressure (BP) screening, follow-up, and treatment before and during the coronavirus disease 2019 (COVID-19) pandemic. Methods: Retrospective cohort study using electronic medical records from the Canadian Primary Care Sentinel Surveillance Network to capture paediatric visits (aged 3 to 18) between January 1, 2011, and December 31, 2020. Time-series analysis was performed using documentation of monthly BP, high BP, follow-up of abnormal BP, and antihypertensive prescribing. We assessed differences between pre (January 1, 2011 to March 11, 2020) and during COVID-19 (March 12, 2020 to December 31, 2020). Results: Of 343,191 paediatric patients, 30.9% had ≥1 paediatric BP documented. Documentation of BP increased each year from 17.3% in 2011 to 19.8% in 2019 (β = 0.05, 95% CI 0.04, 0.07, P < 0.001), with a decrease in trend in 2020 to 11.0% (β = −16.95, 95% CI −18.91, −14.99, P < 0.001). There was an increasing pre-pandemic trend for laboratory screening and prescribing (β = 0.12, 95% CI 0.1, 0.14, P < 0.0001; β = 0.02, 95% CI 0.02, 0.02, P < 0.0001). During the COVID-19 pandemic, laboratory screening further increased (24.5% to 31.1%; β = 5.19, 95% CI 2.03, 8.35, P = 0.002), whereas there was no significant change in prescribing trends (1.3% to 1.4%; β = 0.15, 95% CI −0.01, 0.32, P = 0.07). Conclusions: Documentation of BP increased annually, then declined precipitously during the COVID-19 pandemic. Despite lower BP screening and follow-up, the prevalence of hypertension and antihypertensive prescribing remained stable. Clinical practice trends in primary care highlight areas to improve the care and management of hypertensive paediatric patients.

Wait times at Canadian multidisciplinary pain clinics have been reported as excessive for nearly 2 decades. The aim of this study was to gain insight into the patient experience of waiting for chronic pain specialty care. A cross-sectional survey of new patients waiting for an appointment was conducted in six multidisciplinary pain clinics, including one pediatric clinic, in Ontario, Quebec, and Manitoba between February 2020 and October 2022. Participants were asked about the length of time they waited for their appointment since being referred, their quality of life, health care professionals seen while waiting, and an open-ended question, "Is there anything else you'd like to tell us?" Among the 493 adult and 100 pediatric respondents, 53% of adults and 82% of children reported wait times under 6 months, whereas 22% of adults and 4% of children waited longer than a year. Between 52% and 63% of adults and 29% to 48% of children reported being affected by chronic pain "quite a bit" or "extremely" on measures of quality of life. The most visited health care professionals while waiting for a pain clinic appointment were family doctors/nurse practitioners for adults and physiotherapists for children. Qualitative analysis of open-ended question responses revealed eight themes: system navigation issues, administrative issues, decreased quality of life, distress, self-advocacy, coping strategies, communication, and distrust. Our findings provide real-time regional snapshots into the impact of long wait times experienced by Canadians living with chronic pain. There is an urgent need to better support patients during the waiting period. Expanding technologies such as electronic consultation hold great promise.

The rapid deployment of virtual primary care visits served as a first-line response to COVID-19 and can now be examined for insights, particularly as virtual care is playing an ongoing role in patient care and consultations. Input from primary care providers directly responsible for virtual care delivery is needed to inform policies and strategies for quality care and interactions. The overarching goal of this research study was to examine the use of virtual care as a mechanism for primary healthcare delivery. A phenomenological approach investigated the shift in primary care service delivery as experienced by primary care providers and initiated during the COVID-19 pandemic. Focus groups were conducted with primary care providers (n = 21) recruited through email, advertisements, and professional organizations, exploring how virtual care was delivered, the benefits and challenges, workflow considerations, and recommendations for future use. Integrating virtual care was performed with a great deal of autonomy as well as responsibility, and overwhelmingly depended on the telephone. Technology, communication, and workflow flexibility are three key operational aspects of virtual care and its delivery. Providers highlighted cross-cutting themes related to the dynamics of virtual care including balancing risk for quality care, physician work/life balance, efficiency, and patient benefits. Primary care providers felt that virtual care options allowed increased flexibility to attend to the needs of patients and manage their practice workload, and a few scenarios were shared for when virtual care might be best suited. However, they also recognized the need to balance in-person and virtual visits, which may require guidelines that support navigating various levels of care. Overall, virtual care was considered a good addition to the whole ‘care package’ but continued development and refinement is an expectation for optimizing and sustaining future use.

Background: To validate case definitions for eczema using primary care Electronic Medical Record (EMR) data from the Canadian Primary Care Sentential Surveillance Network (CPCSSN). Methods: This study used EMR data from 1,574 primary care providers in seven Canadian provinces, representing 689,301 patients. Using a subset of patient records seven medical students or family medicine residents created a reference set of 1,772 patients. A total of 23 clinician-informed case definitions were validated against the reference. We assessed agreement using sensitivity (SE), specificity (SP), positive predictive value (PPV), negative predictive value (NPV) and overall accuracy. The case definitions with the best agreement statistics were deployed to estimate the prevalence of eczema in the CPCSSN. Results: Case definition 1 had the highest SE (92.1%,85.0-96.5) but a lower SP (88.5%,86.7–90.1) and PPV (36.6%,33.1–40.3). Case definition 7 was the most specific case definition with a SP (99.8%, 99.4–100) and PPV (84.2%,61.2–94.7) but low SE (15.8%,9.3–24.5). Case definition 17 had a SE (75.3%, 65.7–83.3), SP (93.8%, 91.5–94.3) and PPV 43.7% (38.3–49.2). When we applied the most specific and most sensitive case definitions, we estimate the prevalence of eczema to be between 0.8 and 15.1%. Case definition 17 suggests an eczema prevalence estimate of 8.2% (8.08–8.21%). Conclusions: We validated EMR-based eczema case definitions to estimate the prevalence of clinician-documented eczema. Future studies may choose to apply one or more of these definitions’ dependent on their studies objectives to inform disease surveillance as well as explore burden of illness or interventions related to eczema care in Canada.