Dongfang Xu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Dongfang Xu has a total of 103 co-authors. Here are the top co-authors by number of shared publications:
- Graciela Gonzalez-Hernandez
Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA
9 shared publications - Steven Bethard
University of Arizona
6 shared publications - Guillermo López-García
Department of Computational Biomedicine, Cedars-Sinai Medical Center
5 shared publications - Karen O'COnnor
Perelman School of Medicine, University of Pennsylvania
4 shared publications - Davy Weissenbacher
Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA
3 shared publications Ivan Flores Amaro
Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA USA
3 shared publications
Latest Publications
- Open Access
An LLM enabled real-time estimation of seasonal influenza vaccine effectiveness from social media data
Influenza vaccine effectiveness (VE) is estimated from a limited number of clinics using a test-negative design. These standard estimates face geographic, temporal, and operational constraints. Using Twitter/X data, we applied few-shot chain-of-thought prompting to identify self-reported vaccination status and influenza test results, then implemented a test-negative-like design to estimate VE. Our estimates fell within the range of interim reports and could complement current systems, improving feasibility, timeliness, and scalability.
Open AccessAn LLM enabled real-time estimation of seasonal influenza vaccine effectiveness from social media data
Influenza vaccine effectiveness (VE) is estimated from a limited number of clinics using a test-negative design. These standard estimates face geographic, temporal, and operational constraints. Using Twitter/X data, we applied few-shot chain-of-thought prompting to identify self-reported vaccination status and influenza test results, then implemented a test-negative-like design to estimate VE. Our estimates fell within the range of interim reports and could complement current systems, improving feasibility, timeliness, and scalability.
- 14 December 2025
- Published by World Scientific Pub Co Pte Ltd in Biocomputing 2020
- Vol. 31, 71-84
Scoring Physician Risk Communication in Prostate Cancer Using Large Language Models
Effective risk communication is essential to shared decision-making in prostate cancer care. However, the quality of physician communication of key concepts varies widely in real-world consultations. Manual evaluation of communication is labor-intensive and not scalable. We present a structured, rubric-based framework that uses large language models (LLMs) to automatically score the quality of risk communication in prostate cancer consultations. Using transcripts from 20 clinical visits, we curated and annotated 487 physician-spoken sentences that referenced five key concepts for shared decision-making: cancer prognosis, life expectancy, and three treatment side effects (erectile dysfunction, incontinence, and irritative urinary symptoms). Each sentence was assigned a score from 0 to 5 based on the precision and patient-specificity of communicated risk, using a validated scoring rubric. We modeled this task as five multiclass classification problems and evaluated both finetuned transformer baselines and GPT-4o with rubric-based and chain-of-thought (CoT) prompting. Our best performing approach, which combined rubric-based CoT prompting with few-shot learning, achieved micro averaged F1 scores between 85.0 and 92.0 across domains, outperforming supervised baselines and matching inter-annotator agreement. These findings establish a scalable foundation for AI-driven evaluation of physician-patient communication in oncology and beyond.
- 14 December 2025
- Published by World Scientific Pub Co Pte Ltd in Biocomputing 2020
- Vol. 31, 71-84
Scoring Physician Risk Communication in Prostate Cancer Using Large Language Models
Effective risk communication is essential to shared decision-making in prostate cancer care. However, the quality of physician communication of key concepts varies widely in real-world consultations. Manual evaluation of communication is labor-intensive and not scalable. We present a structured, rubric-based framework that uses large language models (LLMs) to automatically score the quality of risk communication in prostate cancer consultations. Using transcripts from 20 clinical visits, we curated and annotated 487 physician-spoken sentences that referenced five key concepts for shared decision-making: cancer prognosis, life expectancy, and three treatment side effects (erectile dysfunction, incontinence, and irritative urinary symptoms). Each sentence was assigned a score from 0 to 5 based on the precision and patient-specificity of communicated risk, using a validated scoring rubric. We modeled this task as five multiclass classification problems and evaluated both finetuned transformer baselines and GPT-4o with rubric-based and chain-of-thought (CoT) prompting. Our best performing approach, which combined rubric-based CoT prompting with few-shot learning, achieved micro averaged F1 scores between 85.0 and 92.0 across domains, outperforming supervised baselines and matching inter-annotator agreement. These findings establish a scalable foundation for AI-driven evaluation of physician-patient communication in oncology and beyond.
- Open Access
Scoring Physician Risk Communication in Prostate Cancer Using Large Language Models
Effective risk communication is essential to shared decision-making in prostate cancer care. However, the quality of physician communication of key tradeoffs varies widely in real-world consultations. Manual evaluation of communication is labor-intensive and not scalable. We present a structured, rubric-based framework that uses large language models (LLMs) to automatically score the quality of risk communication in prostate cancer consultations. Using transcripts from 20 clinical visits, we curated and annotated 487 physician-spoken sentences that referenced five decision-making domains: cancer prognosis, life expectancy, and three treatment side effects (erectile dysfunction, incontinence, and irritative urinary symptoms). Each sentence was assigned a score from 0 to 5 based on the precision and patient-specificity of communicated risk, using a validated scoring rubric. We modeled this task as five multiclass classification problems and evaluated both fine-tuned transformer baselines and GPT-4o with rubric-based and chain-of-thought (CoT) prompting. Our best performing approach, which combined rubric-based CoT prompting with few-shot learning, achieved micro averaged F1 scores between 85.0 and 92.0 across domains, outperforming supervised baselines and matching inter-annotator agreement. These findings establish a scalable foundation for AI-driven evaluation of physician–patient communication in oncology and beyond.
Open AccessScoring Physician Risk Communication in Prostate Cancer Using Large Language Models
Effective risk communication is essential to shared decision-making in prostate cancer care. However, the quality of physician communication of key tradeoffs varies widely in real-world consultations. Manual evaluation of communication is labor-intensive and not scalable. We present a structured, rubric-based framework that uses large language models (LLMs) to automatically score the quality of risk communication in prostate cancer consultations. Using transcripts from 20 clinical visits, we curated and annotated 487 physician-spoken sentences that referenced five decision-making domains: cancer prognosis, life expectancy, and three treatment side effects (erectile dysfunction, incontinence, and irritative urinary symptoms). Each sentence was assigned a score from 0 to 5 based on the precision and patient-specificity of communicated risk, using a validated scoring rubric. We modeled this task as five multiclass classification problems and evaluated both fine-tuned transformer baselines and GPT-4o with rubric-based and chain-of-thought (CoT) prompting. Our best performing approach, which combined rubric-based CoT prompting with few-shot learning, achieved micro averaged F1 scores between 85.0 and 92.0 across domains, outperforming supervised baselines and matching inter-annotator agreement. These findings establish a scalable foundation for AI-driven evaluation of physician–patient communication in oncology and beyond.
- Open Access
Automated Insomnia Phenotyping from Electronic Health Records: Leveraging Large Language Models to Decode Clinical Narratives
Insomnia is a highly prevalent but often underdiagnosed condition in clinical practice. Its inconsistent documentation in electronic health records (EHRs) limits population-level analyses and obstructs efforts to evaluate treatment patterns or outcomes. We present a novel, fully automated approach for phenotyping insomnia directly from unstructured clinical notes using generative large language models (LLMs). Leveraging prompt engineering with few-shot learning and chain-of-thought reasoning, we evaluated our system on two distinct corpora: inpatient clinical notes from MIMIC-III and outpatient primary care notes from the University of Kansas Health System (KUMC). Our models—Llama 70B and Llama 405B—achieved F1 scores of 93.0 on the MIMIC corpus and 85.7 on the KUMC corpus, substantially outperforming domain-adapted BERT-based classifiers. Ultimately, our framework offers a scalable and interpretable solution for clinical phenotyping of insomnia and can serve as a blueprint for similar efforts targeting other underdiagnosed or under-documented conditions in the EHR.
Open AccessAutomated Insomnia Phenotyping from Electronic Health Records: Leveraging Large Language Models to Decode Clinical Narratives
Insomnia is a highly prevalent but often underdiagnosed condition in clinical practice. Its inconsistent documentation in electronic health records (EHRs) limits population-level analyses and obstructs efforts to evaluate treatment patterns or outcomes. We present a novel, fully automated approach for phenotyping insomnia directly from unstructured clinical notes using generative large language models (LLMs). Leveraging prompt engineering with few-shot learning and chain-of-thought reasoning, we evaluated our system on two distinct corpora: inpatient clinical notes from MIMIC-III and outpatient primary care notes from the University of Kansas Health System (KUMC). Our models—Llama 70B and Llama 405B—achieved F1 scores of 93.0 on the MIMIC corpus and 85.7 on the KUMC corpus, substantially outperforming domain-adapted BERT-based classifiers. Ultimately, our framework offers a scalable and interpretable solution for clinical phenotyping of insomnia and can serve as a blueprint for similar efforts targeting other underdiagnosed or under-documented conditions in the EHR.
- Open Access
Detecting Medication Mentions in Social Media Data Using Large Language Models
The automatic extraction of medication mentions from social media data is critical for pharmacovigilance and public health monitoring. In this study, we present an end-to-end generative approach based on instruction-tuned large language models (LLMs) for medication mention extraction from Twitter. Reformulating the task as a text-to-text generation problem, our models achieve state-of-the-art results on both fine-grained span extraction and coarse-grained tweet-level classification, surpassing traditional sequence labeling baselines and previous best-performing systems. We demonstrate that fine-tuning Flan-T5 models enables efficient and accurate extraction while simplifying the architecture by eliminating complex multi-stage pipelines. Additionally, we show that lexicon-based filtering further improves performance by reducing false positives. Our models are publicly available, providing high-performing and efficient tools for large-scale pharmacological analysis of social media data.
Open AccessDetecting Medication Mentions in Social Media Data Using Large Language Models
The automatic extraction of medication mentions from social media data is critical for pharmacovigilance and public health monitoring. In this study, we present an end-to-end generative approach based on instruction-tuned large language models (LLMs) for medication mention extraction from Twitter. Reformulating the task as a text-to-text generation problem, our models achieve state-of-the-art results on both fine-grained span extraction and coarse-grained tweet-level classification, surpassing traditional sequence labeling baselines and previous best-performing systems. We demonstrate that fine-tuning Flan-T5 models enables efficient and accurate extraction while simplifying the architecture by eliminating complex multi-stage pipelines. Additionally, we show that lexicon-based filtering further improves performance by reducing false positives. Our models are publicly available, providing high-performing and efficient tools for large-scale pharmacological analysis of social media data.
- Open Access
Mining Social Media Data for Influenza Vaccine Effectiveness Using a Large Language Model and Chain-of-Thought Prompting
Influenza vaccine effectiveness (VE) estimation plays a critical role in public health decision-making by quantifying the real-world impact of vaccination campaigns and guiding policy adjustments. Current approaches to VE estimation are constrained by limited population representation, selection bias, and delayed reporting. To address some of these gaps, we propose leveraging large language models (LLMs) with few-shot chain-of-thought (CoT) prompting to mine social media data for real-time influenza VE estimation. We annotated over 4,000 tweets from the 2020–2021 flu season using structured guidelines, achieving high inter-annotator agreement. Our best prompting strategy achieves F 1 scores above 87% for identifying influenza vaccination status and test outcomes, outperforming traditional supervised fine-tuning methods by large margins. These findings indicate that LLM-based prompting approaches effectively identify relevant social media information for influenza VE estimation, offering a valuable real-time surveillance tool that complements traditional epidemiological methods.
Open AccessMining Social Media Data for Influenza Vaccine Effectiveness Using a Large Language Model and Chain-of-Thought Prompting
Influenza vaccine effectiveness (VE) estimation plays a critical role in public health decision-making by quantifying the real-world impact of vaccination campaigns and guiding policy adjustments. Current approaches to VE estimation are constrained by limited population representation, selection bias, and delayed reporting. To address some of these gaps, we propose leveraging large language models (LLMs) with few-shot chain-of-thought (CoT) prompting to mine social media data for real-time influenza VE estimation. We annotated over 4,000 tweets from the 2020–2021 flu season using structured guidelines, achieving high inter-annotator agreement. Our best prompting strategy achieves F 1 scores above 87% for identifying influenza vaccination status and test outcomes, outperforming traditional supervised fine-tuning methods by large margins. These findings indicate that LLM-based prompting approaches effectively identify relevant social media information for influenza VE estimation, offering a valuable real-time surveillance tool that complements traditional epidemiological methods.
- Open Access
- 13 January 2024
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 31 (4) , 991-996
Overview of the 8th Social Media Mining for Health Applications (#SMM4H) shared tasks at the AMIA 2023 Annual Symposium
Objective: The aim of the Social Media Mining for Health Applications (#SMM4H) shared tasks is to take a community-driven approach to address the natural language processing and machine learning challenges inherent to utilizing social media data for health informatics. In this paper, we present the annotated corpora, a technical summary of participants’ systems, and the performance results. Methods: The eighth iteration of the #SMM4H shared tasks was hosted at the AMIA 2023 Annual Symposium and consisted of 5 tasks that represented various social media platforms (Twitter and Reddit), languages (English and Spanish), methods (binary classification, multi-class classification, extraction, and normalization), and topics (COVID-19, therapies, social anxiety disorder, and adverse drug events). Results: In total, 29 teams registered, representing 17 countries. In general, the top-performing systems used deep neural network architectures based on pre-trained transformer models. In particular, the top-performing systems for the classification tasks were based on single models that were pre-trained on social media corpora. Conclusion: To facilitate future work, the datasets—a total of 61 353 posts—will remain available by request, and the CodaLab sites will remain active for a post-evaluation phase.
- 13 January 2024
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 31 (4) , 991-996
Open AccessOverview of the 8th Social Media Mining for Health Applications (#SMM4H) shared tasks at the AMIA 2023 Annual Symposium
Objective: The aim of the Social Media Mining for Health Applications (#SMM4H) shared tasks is to take a community-driven approach to address the natural language processing and machine learning challenges inherent to utilizing social media data for health informatics. In this paper, we present the annotated corpora, a technical summary of participants’ systems, and the performance results. Methods: The eighth iteration of the #SMM4H shared tasks was hosted at the AMIA 2023 Annual Symposium and consisted of 5 tasks that represented various social media platforms (Twitter and Reddit), languages (English and Spanish), methods (binary classification, multi-class classification, extraction, and normalization), and topics (COVID-19, therapies, social anxiety disorder, and adverse drug events). Results: In total, 29 teams registered, representing 17 countries. In general, the top-performing systems used deep neural network architectures based on pre-trained transformer models. In particular, the top-performing systems for the classification tasks were based on single models that were pre-trained on social media corpora. Conclusion: To facilitate future work, the datasets—a total of 61 353 posts—will remain available by request, and the CodaLab sites will remain active for a post-evaluation phase.
- Open Access
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 20780-20793
Automatic sentence segmentation of clinical record narratives in real-world data
Dongfang Xu, Davy Weissenbacher, Karen O’Connor, Siddharth Rawal, Graciela Gonzalez Hernandez. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 20780-20793
Open AccessAutomatic sentence segmentation of clinical record narratives in real-world data
Dongfang Xu, Davy Weissenbacher, Karen O’Connor, Siddharth Rawal, Graciela Gonzalez Hernandez. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.
- Open Access
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 183-195
Overview of the 9th Social Media Mining for Health Applications (#SMM4H) Shared Tasks at ACL 2024 – Large Language Models and Generalizability for Social Media NLP
Dongfang Xu, Guillermo Garcia, Lisa Raithel, Philippe Thomas, Roland Roller, Eiji Aramaki, Shoko Wakamiya, Shuntaro Yada, Pierre Zweigenbaum, Karen O’Connor, Sai Samineni, Sophia Hernandez, Yao Ge, Swati Rajwal, Sudeshna Das, Abeed Sarker, Ari Klein, Ana Schmidt, Vishakha Sharma, Raul Rodriguez-Esteban, Juan Banda, Ivan Amaro, Davy Weissenbacher, Graciela Gonzalez-Hernandez. Proceedings of The 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks. 2024.
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 183-195
Open AccessOverview of the 9th Social Media Mining for Health Applications (#SMM4H) Shared Tasks at ACL 2024 – Large Language Models and Generalizability for Social Media NLP
Dongfang Xu, Guillermo Garcia, Lisa Raithel, Philippe Thomas, Roland Roller, Eiji Aramaki, Shoko Wakamiya, Shuntaro Yada, Pierre Zweigenbaum, Karen O’Connor, Sai Samineni, Sophia Hernandez, Yao Ge, Swati Rajwal, Sudeshna Das, Abeed Sarker, Ari Klein, Ana Schmidt, Vishakha Sharma, Raul Rodriguez-Esteban, Juan Banda, Ivan Amaro, Davy Weissenbacher, Graciela Gonzalez-Hernandez. Proceedings of The 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks. 2024.
- Open Access
- 1 January 2023
- Published by Oxford University Press (OUP) in Database: The Journal of Biological Databases and Curation
- Vol. 2023
Automatic Extraction of Medication Mentions from Tweets—Overview of the BioCreative VII Shared Task 3 Competition
This study presents the outcomes of the shared task competition BioCreative VII (Task 3) focusing on the extraction of medication names from a Twitter user’s publicly available tweets (the user’s ‘timeline’). In general, detecting health-related tweets is notoriously challenging for natural language processing tools. The main challenge, aside from the informality of the language used, is that people tweet about any and all topics, and most of their tweets are not related to health. Thus, finding those tweets in a user’s timeline that mention specific health-related concepts such as medications requires addressing extreme imbalance. Task 3 called for detecting tweets in a user’s timeline that mentions a medication name and, for each detected mention, extracting its span. The organizers made available a corpus consisting of 182 049 tweets publicly posted by 212 Twitter users with all medication mentions manually annotated. The corpus exhibits the natural distribution of positive tweets, with only 442 tweets (0.2%) mentioning a medication. This task was an opportunity for participants to evaluate methods that are robust to class imbalance beyond the simple lexical match. A total of 65 teams registered, and 16 teams submitted a system run. This study summarizes the corpus created by the organizers and the approaches taken by the participating teams for this challenge. The corpus is freely available at https://biocreative.bioinformatics.udel.edu/tasks/biocreative-vii/track-3/. The methods and the results of the competing systems are analyzed with a focus on the approaches taken for learning from class-imbalanced data.
- 1 January 2023
- Published by Oxford University Press (OUP) in Database: The Journal of Biological Databases and Curation
- Vol. 2023
Open AccessAutomatic Extraction of Medication Mentions from Tweets—Overview of the BioCreative VII Shared Task 3 Competition
This study presents the outcomes of the shared task competition BioCreative VII (Task 3) focusing on the extraction of medication names from a Twitter user’s publicly available tweets (the user’s ‘timeline’). In general, detecting health-related tweets is notoriously challenging for natural language processing tools. The main challenge, aside from the informality of the language used, is that people tweet about any and all topics, and most of their tweets are not related to health. Thus, finding those tweets in a user’s timeline that mention specific health-related concepts such as medications requires addressing extreme imbalance. Task 3 called for detecting tweets in a user’s timeline that mentions a medication name and, for each detected mention, extracting its span. The organizers made available a corpus consisting of 182 049 tweets publicly posted by 212 Twitter users with all medication mentions manually annotated. The corpus exhibits the natural distribution of positive tweets, with only 442 tweets (0.2%) mentioning a medication. This task was an opportunity for participants to evaluate methods that are robust to class imbalance beyond the simple lexical match. A total of 65 teams registered, and 16 teams submitted a system run. This study summarizes the corpus created by the organizers and the approaches taken by the participating teams for this challenge. The corpus is freely available at https://biocreative.bioinformatics.udel.edu/tasks/biocreative-vii/track-3/. The methods and the results of the competing systems are analyzed with a focus on the approaches taken for learning from class-imbalanced data.