QPQuynh Pham

Quynh Pham

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Quynh Pham has a total of 214 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Retrieval-Augmented Generation for Medical Question Answering on a Heart Failure Dataset: Performance Analysis

    by Shiran Zhang, Evelyn Phan, Pedro Velmovitsky, Quynh Pham, Scott Sanner

    Background: The integration of retrieval-augmented generation (RAG) systems into the domain of medical question-answering (QA) presents a significant opportunity to enhance the effectiveness and accuracy of clinical support systems. Objective: This study aimed to explore the design choices within the RAG framework and the use of large language model (LLM) classifiers to optimize medical QA systems, enhancing response quality for patient and caregiver queries of varying risk levels. Methods: In total, we curated a dataset of 109 patient and caregiver questions related to heart failure (HF)—categorized into answerable (direct, fact-based queries), helpful deferral (general guidance or lifestyle advisory queries), and nonanswerable (out-of-scope or high-risk and medical intervention queries) types—along with relevant documents and a target answer for each question from the website The Heart Hub . Applying a system architecture leveraging RAG with a structured query taxonomy and robust classification mechanisms, this paper provided an empirical assessment for medical QA on a HF dataset and introduced a QA system pipeline design, providing a foundation for extended application across various medical fields. Specifically, we evaluated design choices in the initial retrieval stage of RAG and their impact on performance. We assessed final answer quality from the generation stage using popular passage scoring methods for QA, such as Recall-Oriented Understudy for Gisting Evaluation (ROUGE), BERTScore, and Intersection over Union score. Results: The pipeline first uses an LLM-based classifier, achieving 65% accuracy for answerable and helpful deferral queries and 100% accuracy for identifying nonanswerable queries. In information retrieval, the BioMedical Contrastive Pre-trained Transformers (MedCPT) cross encoder performed best as a dense retrieval method, delivering an average of 93% recall @ 7 through ranked relevance scores to obtain the top documents with recall @ k denoting recall computed over the top-k retrieved items. For further retrieving snippets from such documents, its average performance was 72.5% for sentence-level snippets and 83% for paragraph-level snippets. A second LLM-based classifier, used to refine the generated responses, resulted in an overall reduction in ROUGE-1 recall by 13% and Bidirectional Encoder Representations from Transformers (BERT) precision by 11%. However, Intersection over Union scores, or the overlap between “gold answers” and system answers, increased by 24%, demonstrating enhanced alignment with ground truth responses. This also indicates the system’s improved ability to generate concise and accurate medical responses. Conclusions: The implementation of a structured RAG framework paired with LLM classifiers for medical QA introduces a promising avenue for enhancing clinical decision support systems. By systematically analyzing the impact of query taxonomy, retrieval configurations, and response strategies, this approach clarifies the relative importance of each component within the medical RAG system using a HF dataset. Our findings provide actionable guidance on optimal design choices for maximizing retrieval and response accuracy; thus, informing the development of robust, scalable medical QA systems.


  • The transformative potential of digital therapeutics in pediatrics

    by Vina Mohabir, Chitra Lalloo, Andrew M. Briggs, Christopher Eccleston, Courtney Hess, Quynh Pham, Helen Slater, Pedro Velmovitsky, Jennifer N. Stinson

    Digital therapeutics (DTx) are applied healthcare technologies designed to prevent, manage, or treat health conditions. In pediatric pain, DTx can enhance access to evidence-based, effective pain assessment and management. This article explores technologies such as artificial intelligence (AI), extended reality (XR), mobile health (mHealth), and sensors (eg, smartwatches). Equitable DTx deployment can address geographic disparities by enabling on-demand pain care, necessitating an intelligent learning DTx-enabled health system. Ensuring safety will require comprehensive regulatory frameworks at national and international levels. Achieving this transformation requires robust regulatory frameworks, workforce training, and equitable codesign to ensure accessible, evidence-based care for all pediatric populations.


  • Open Access

    Wearable Technologies in Head and Neck Oncology: Scoping Review

    by Matthew Shammas-Toma, Gianluca Sampieri, Michael Xie, Aishwaria Maxwell, Alex Esemezie, Quynh Pham, Joseph A Cafazzo, Philip Wong, C Jillian Tsai, David P Goldstein, et al.

    Background: Head and neck cancer (HNC) survivors face profound functional and quality-of-life deficits due to disease- and treatment-related sequelae, ranging from mild fatigue to debilitating dysphagia. Wearable technology, by monitoring biometric data such as step counts or providing swallowing biofeedback, offers a unique method for tracking and monitoring the negative effects of HNC. Objective: The aim of this study was to explore the current applications of wearable technology in HNC. Methods: A scoping review was conducted following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. A search strategy was built, and a literature search was performed across 5 databases. The initial search yielded 5256 studies, which underwent a 2-phase screening process: title and abstract review followed by full-text review. Inclusion criteria included peer-reviewed, English-language articles published between January 2002 and April 2024 that used wearable technology in HNC care. After full-text review, 9 studies met the inclusion criteria. Data were manually extracted and synthesized narratively. Results: The included studies examined 3 main types of wearable devices: radioactivity (2 studies), physical activity (4 studies), and throat physiology monitors (3 studies). Radioactivity monitors detected residual radioactivity and thyroidal radioiodine uptake. They demonstrated potential to reduce radioactivity exposure risk and personalize radiation doses for patients with thyroid cancer. Physical activity monitors tracked step counts, heart rate, and sleep habits. Low step counts were significantly associated with increased anxiety, radiation-related toxicity, hospital admission rates, and feeding tube placement. One study also linked poor sleep patterns to declines in quality of life. Throat physiology monitors measured pharyngeal electromyography data as well as extrinsic laryngeal muscle movements. Throat sensors achieved high accuracy in classifying swallowing events and translating muscle movements into speech. While earliest in the development continuum, they are promising tools for swallowing and vocal rehabilitation therapy. Barriers to wearable adoption included wearable discomfort, technical difficulties, and patient withdrawal due to treatment side effects. As the definition of wearable adherence varied widely, we propose that future studies report wearable adherence as “percentage of prescribed wear time achieved” to facilitate cross-study comparisons. Conclusions: Wearable technology may enhance treatment monitoring, prognostication, and rehabilitation in head and neck oncology. Radioactivity and physical activity monitors provide actionable insights for clinical decision-making, while throat physiology monitors offer innovative solutions for speech and swallowing therapy. However, challenges such as device adherence, data integration, and patient comfort must be addressed to realize their full potential. Future research should prioritize larger, longitudinal studies, standardized adherence metrics, and consider the integration of artificial intelligence to refine predictive capabilities. By overcoming these barriers, wearable technology could transform survivorship care, improving functional outcomes and quality of life for patients with HNC.


  • Open Access

    Retrieval-Augmented Generation for Medical Question Answering on a Heart Failure Dataset: Performance Analysis (Preprint)

    by Shiran Zhang, Evelyn Phan, Pedro Velmovitsky, Quynh Pham, Scott Sanner

    BACKGROUND The integration of Retrieval-Augmented Generation (RAG) systems into the domain of medical question-answering (QA) presents a significant opportunity to enhance the effectiveness and accuracy of clinical support systems. OBJECTIVE This study aimed to explore the design choices within the RAG framework and the use of large language model (LLM) classifiers to optimize medical QA systems, enhancing response quality for patient and caregiver queries of varying risk levels. METHODS In total, we curated a dataset of 109 patient and caregiver questions related to heart failure–categorized into answerable (direct, fact-based queries), helpful deferral (general guidance or lifestyle advisory queries), and non-answerable (out-of-scope or high-risk/medical intervention queries) types–along with relevant documents and a target answer for each question from the website The Heart Hub. Applying a system architecture leveraging RAG with a structured query taxonomy and robust classification mechanisms, this paper provided an empirical assessment for medical QA on a heart failure dataset and introduced a question-answering system pipeline design, providing a foundation for extended application across various medical fields. Specifically, we evaluated design choices in the initial retrieval stage of RAG and their impact on performance. We assessed final answer quality from the generation stage using popular passage scoring methods for QA, such as ROUGE, BERTScore, and Intersection over Union (IOU) score. RESULTS The pipeline first employs an LLM-based classifier, achieving 65% accuracy for answerable and helpful deferral queries and 100% accuracy for identifying non-answerable queries. In information retrieval (IR), the bioMedical Contrastive Pre-trained Transformers (MedCPT) cross encoder performed best as a dense retrieval method, delivering an average of 93% recall @ 7 through ranked relevance scores to obtain the top documents. For further retrieving snippets from such documents, its average performance was 72.5% for sentence-level snippets and 83% for paragraph-level snippets. A second LLM-based classifier, used to refine the generated responses, resulted in an overall reduction in ROUGE-1 recall by 13% and BERT precision by 11%. However, IoU scores, or the overlap between “gold answers” and system answers, increased by 24%, demonstrating enhanced alignment with ground truth responses. This also indicates the system’s improved ability to generate concise and accurate medical responses. CONCLUSIONS The implementation of a structured RAG framework paired with LLM classifiers for medical QA introduces a promising avenue for enhancing clinical decision support systems. By systematically analyzing the impact of query taxonomy, retrieval configurations, and response strategies, this approach clarifies the relative importance of each component within the medical RAG system using a heart failure dataset. Our findings provide actionable guidance on optimal design choices for maximizing retrieval and response accuracy, thus informing the development of robust, scalable medical QA systems.

  • Evaluating Digitally Enabled Health Services

    by Carolyn Steele Gray, Nelson Shen, Quynh Pham, David Wiljer


  • Open Access

    Mental Health Screening Tools for Anxiety and Depression in Post-Secondary Students: A Literature Review to Inform Digital Intervention Development (Preprint)

    by Joseph Saliba, Xinyi Liu, Pedro Elkind Velmovitsky, Julia Hews-Girard, Scott Patten, Nathan King, Charles Keown-Stoneman, Anne Duffy, Quynh Pham

    BACKGROUND Post-secondary students experience increasingly high rates of mental health challenges. Evidence-supported digital mental health interventions (DMHIs) could form part of an integrated, sustainable prevention and early intervention approach suited for university students. Screening tools like the two- and nine-item Patient Health Questionnaire (PHQ-2, PHQ-9) and the two- and seven-item Generalized Anxiety Disorder questionnaire (GAD-2, GAD-7) could inform individualized signposting to appropriate support resources and services, yet their validation has been largely in general adult populations, raising concerns about their suitability in post-secondary student contexts. OBJECTIVE To bridge this gap, this literature review aims to summarize the psychometric properties of commonly used screening tools, particularly the PHQ-9, PHQ-2, GAD-7, and GAD-2, across student and general adult populations. Proper use and population specific screening thresholds are critical to develop evidence-based DMHIs that accurately assess, monitor, visualize, and respond to student mental health challenges. METHODS Following PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines, a search of peer-reviewed literature across PubMed, Web of Science, and PsychINFO yielded 82 studies, 49 of which were selected for data extraction and analysis. Studies were clustered into Group 1 (student populations) and Group 2 (general adult populations). This review extracted optimal screening thresholds along with sensitivity, specificity, area under the curve, and predictive values. RESULTS Psychometric properties varied between student and general adult samples. Among students, GAD-2 showed the highest sensitivity (94.2%) and highest specificity (85.2%), while PHQ-2 demonstrated weaker specificity (66.5%). Optimal screening thresholds for student populations were generally equal or lower than those used in general adult settings, especially for the GAD-7 (cut-off: 6 vs. 10). Missing and inconsistent reporting on ethnicity and gender differences limited equity-oriented analysis. CONCLUSIONS Applying general adult screening thresholds may compromise accuracy and care equity in student populations. Student-calibrated screening thresholds are important to improve screening precision for DMHI utility and effectiveness.

  • Open Access

    Application of a Sociotechnical Framework to Uncover Factors That Influence Effective User Engagement With Digital Mental Health Tools in Clinical Care Contexts: Scoping Review

    by Brian Lo, Keri Durocher, Rebecca Charow, Sarah Kimball, Quynh Pham, Sanjeev Sockalingam, David Wiljer, Gillian Strudwick

    Background: Digital health tools such as mobile apps and patient portals continue to be embedded in clinical care pathways to enhance mental health care delivery and achieve the quintuple aim of improving patient experience, population health, care team well-being, health care costs, and equity. However, a key issue that has greatly hindered the value of these tools is the suboptimal user engagement by patients and families. With only a small fraction of users staying engaged over time, there is a great need to better understand the factors that influence user engagement with digital mental health tools in clinical care settings. Objective: This review aims to identify the factors relevant to user engagement with digital mental health tools in clinical care settings using a sociotechnical approach. Methods: A scoping review methodology was used to identify the relevant factors from the literature. Five academic databases (MEDLINE, Embase, CINAHL, Web of Science, and PsycINFO) were searched to identify pertinent articles using key terms related to user engagement, mental health, and digital health tools. The abstracts were screened independently by 2 reviewers, and data were extracted using a standardized data extraction form. Articles were included if the digital mental health tool had at least 1 patient-facing component and 1 clinician-facing component, and at least one of the objectives of the article was to examine user engagement with the tool. An established sociotechnical framework developed by Sittig and Singh was used to inform the mapping and analysis of the factors. Results: The database search identified 136 articles for inclusion in the analysis. Of these 136 articles, 84 (61.8%) were published in the last 5 years, 47 (34.6%) were from the United States, and 23 (16.9%) were from the United Kingdom. With regard to examining user engagement, the majority of the articles (95/136, 69.9%) used a qualitative approach to understand engagement. From these articles, 26 factors were identified across 7 categories of the established sociotechnical framework. These ranged from technology-focused factors (eg, the modality of the tool) and the clinical environment (eg, alignment with clinical workflows) to system-level issues (eg, reimbursement for physician use of the digital tool with patients). Conclusions: On the basis of the factors identified in this review, we have uncovered how the tool, individuals, the clinical environment, and the health system may influence user engagement with digital mental health tools for clinical care. Future work should focus on validating and identifying a core set of essential factors for user engagement with digital mental health tools in clinical care environments. Moreover, exploring strategies for improving user engagement through these factors would be useful for health care leaders and clinicians interested in using digital health tools in care.


  • Open Access

    Lessons in complexity learned during a Canadian virtual pragmatic trial for prostate cancer survivorship

    by Kaylen J. Pfisterer, Karen Young, Raima Lohani, Ting Xiong, Tiffane Anandarajan, Denise Ng, Tina Jiao, Caitlin Nunn, Denise Bryant-Lukosius, Ricardo Rendon, et al.

  • Open Access

    From Data to Care: Development of a recommendation engine to university student mental health support aligned with stepped care (Preprint)

    by Pedro Velmovitsky, Charles Keown-Stoneman, Kaylen Pfisterer, Julia Hews-Girard, Joseph Saliba, Shumit Saha, Scott Patten, Nathan King, Quynh Pham, Anne Duffy

    BACKGROUND Mental health challenges are prevalent among Canadian higher-education students, with significant rates of depression and anxiety often going untreated due to lack of awareness, stigmatizing beliefs, and practical barriers. The U-Flourish longitudinal survey study launched in 2018 at Queen’s University, repeatedly collecting self-reported data on student mental health and well-being. Objective: Repeated U-Flourish survey data provide a unique opportunity to train an evidence-based recommendation engine to signpost students to rationalized mental health support based on their individual data. OBJECTIVE Repeated U-Flourish survey data provide a unique opportunity to train an evidence-based recommendation engine to signpost students to rationalized mental health support based on their individual data. METHODS Two approaches were used and integrated in developing this recommendation engine: (i) clinically-defined rules by experts in the field which signpost students to indicated levels of care based on clinical factors (i.e., self-harm and suicidal thoughts, symptom levels, lifetime history) following a stepped care approach; (ii) predictive Machine Learning (ML) models, trained with additional data including family history, early adversity, and stress indicators, to predict future risks of clinically significant depression (PHQ-9) and anxiety (GAD-7). Models were created using the XGBoost algorithm, and a 70:30 ratio for training and testing with 10-fold cross-validation. RESULTS Nearly one-third of students were signposted to seek assessment through various levels of university mental health services based on the clinically-defined rules. Optimizing thresholds to reduce false negatives, the ML models predicted worsening anxiety and depression over the year with results comparable to clinical screening (sensitivity equal to or above 90% for all models trained). Individuals considered to be at risk were signposted primarily to self-guided resources for proactive prevention. Model findings also demonstrated that abbreviated screens (PHQ-2, GAD-2), with potential to reduce respondent burden and improve feasibility, can be used without compromising sensitivity. SHAP analyses revealed key predictive factors such as early trauma, family history of mental illness, and functional impairment associated with depression and anxiety symptoms. CONCLUSIONS The system’s dual approach optimizes resource allocation and promotes targeted early intervention and prevention, potentially improving capacity to address the increasing burden on university mental health services. Future directions include independent validation at other institutions, co-design, and validation including framing of AI-powered personalized recommendations with student users to improve accessibility and outcomes.

  • Open Access

    From Data to Care: Development of a recommendation engine to university student mental health support aligned with stepped care (Preprint)

    by Pedro Velmovitsky, Charles Keown-Stoneman, Kaylen Pfisterer, Julia Hews-Girard, Joseph Saliba, Shumit Saha, Scott Patten, Nathan King, Anne Duffy, Quynh Pham

    Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Mental health challenges are prevalent among Canadian higher-education students, with significant rates of depression and anxiety often going untreated, due to reduced early detection, stigmatizing beliefs, and practical barriers. The U-Flourish longitudinal electronic survey study launched in 2018 engages new cohorts of incoming undergraduate students and repeatedly collects data about mental health and well-being and access to support. Objective: U-Flourish survey data provides a unique opportunity to train evidence-based prediction risk models and a personalized recommendation engine to signpost students to indicated mental health support based on their own data. Methods: Two approaches were used and integrated in developing the risk prediction and recommendation engine: (i) clinically-defined rules by experts in the field to detect current and predict the risk of future anxiety and depression and to signpost students to appropriate care using a stepped care approach and based on clinical factors (i.e., self-harm and suicidal thoughts, symptom levels, lifetime history); (ii) Machine Learning (ML) models, trained with additional data including family history, early adversity, and stress indicators, to predict future risks of clinically significant depression (PHQ-9) and anxiety (GAD-7). Models were created using the XGBoost algorithm, and a 70:30 ratio for training and testing with 10-fold cross-validation. Results: 27.5% of students at entry to university from 2018-2023 were identified as having potentially clinically significant levels of anxiety and depression and signposted to university mental health services based on the clinically-defined rules. Optimizing thresholds to reduce false negatives, the ML models predicted anxiety and depression over the year in students screening negative at baseline with accuracy comparable to reported clinical screening as evidenced by sensitivity equal to or above 90% for all models trained). Models had high negative predictive value (89% or above), balanced against low specificity. Individuals identified at risk for anxiety or depression were signposted primarily to self-guided resources supporting proactive prevention. Model findings also demonstrated that abbreviated screens (PHQ-2, GAD-2), with potential to reduce respondent burden and improve adherence, can be used without compromising sensitivity. Indeed, PHQ-2 displayed a 90% sensitivity and GAD-2 a 92% sensitivity. SHAP analyses revealed other predictive factors including childhood trauma, family history of mental illness, and functional impairment associated with reported depression and anxiety symptoms. Conclusions: The risk prediction models and recommendation engine’s dual approach rationalizes support allocation and promotes targeted early intervention and prevention, potentially improving capacity to address the increasing burden on university mental health services. Future directions include further refinement based on a larger harmonized and enriched dataset, independent validation and implementation studies to estimate the complex factors that influence uptake, reach to services and acceptability across more diverse student users.