Haijun Zhai
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Haijun Zhai has a total of 42 co-authors. Here are the top co-authors by number of shared publications:
Todd Lingren
Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA
8 shared publications- Imre Solti
Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA
7 shared publications Qi Li
Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA
6 shared publicationsLaura Stoutenborough
Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA
5 shared publications- Louise Deléger
Université Paris-Saclay, INRAE, MaIAGE
5 shared publications Megan Kaiser
Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA
5 shared publications
Latest Publications
- Open Access
- 1 September 2014
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 21 (5) , 776-784
Phenotyping for patient safety: algorithm development for electronic health record based automated adverse event and medical error detection in neonatal intensive care
Background Although electronic health records (EHRs) have the potential to provide a foundation for quality and safety algorithms, few studies have me
- 1 September 2014
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 21 (5) , 776-784
Open AccessPhenotyping for patient safety: algorithm development for electronic health record based automated adverse event and medical error detection in neonatal intensive care
Background Although electronic health records (EHRs) have the potential to provide a foundation for quality and safety algorithms, few studies have me
- Open Access
- 1 August 2014
- Published by Elsevier in Resuscitation
- Vol. 85 (8) , 1065-1071
Developing and evaluating a machine learning based algorithm to predict the need of pediatric intensive care unit transfer for newly hospitalized children
- 1 August 2014
- Published by Elsevier in Resuscitation
- Vol. 85 (8) , 1065-1071
Open AccessDeveloping and evaluating a machine learning based algorithm to predict the need of pediatric intensive care unit transfer for newly hospitalized children
- Open Access
- 16 July 2014
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 22 (1) , 166-178
Automated clinical trial eligibility prescreening: increasing the efficiency of patient identification for clinical trials in the emergency department
Objectives (1) To develop an automated eligibility screening (ES) approach for clinical trials in an urban tertiary care pediatric emergency department (ED); (2) to assess the effectiveness of natural language processing (NLP), information extraction (IE), and machine learning (ML) techniques on real-world clinical data and trials. Data and methods We collected eligibility criteria for 13 randomly selected, disease-specific clinical trials actively enrolling patients between January 1, 2010 and August 31, 2012. In parallel, we retrospectively selected data fields including demographics, laboratory data, and clinical notes from the electronic health record (EHR) to represent profiles of all 202795 patients visiting the ED during the same period. Leveraging NLP, IE, and ML technologies, the automated ES algorithms identified patients whose profiles matched the trial criteria to reduce the pool of candidates for staff screening. The performance was validated on both a physician-generated gold standard of trial–patient matches and a reference standard of historical trial–patient enrollment decisions, where workload, mean average precision (MAP), and recall were assessed. Results Compared with the case without automation, the workload with automated ES was reduced by 92% on the gold standard set, with a MAP of 62.9%. The automated ES achieved a 450% increase in trial screening efficiency. The findings on the gold standard set were confirmed by large-scale evaluation on the reference set of trial–patient matches. Discussion and conclusion By exploiting the text of trial criteria and the content of EHRs, we demonstrated that NLP-, IE-, and ML-based automated ES could successfully identify patients for clinical trials.
- 16 July 2014
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 22 (1) , 166-178
Open AccessAutomated clinical trial eligibility prescreening: increasing the efficiency of patient identification for clinical trials in the emergency department
Objectives (1) To develop an automated eligibility screening (ES) approach for clinical trials in an urban tertiary care pediatric emergency department (ED); (2) to assess the effectiveness of natural language processing (NLP), information extraction (IE), and machine learning (ML) techniques on real-world clinical data and trials. Data and methods We collected eligibility criteria for 13 randomly selected, disease-specific clinical trials actively enrolling patients between January 1, 2010 and August 31, 2012. In parallel, we retrospectively selected data fields including demographics, laboratory data, and clinical notes from the electronic health record (EHR) to represent profiles of all 202795 patients visiting the ED during the same period. Leveraging NLP, IE, and ML technologies, the automated ES algorithms identified patients whose profiles matched the trial criteria to reduce the pool of candidates for staff screening. The performance was validated on both a physician-generated gold standard of trial–patient matches and a reference standard of historical trial–patient enrollment decisions, where workload, mean average precision (MAP), and recall were assessed. Results Compared with the case without automation, the workload with automated ES was reduced by 92% on the gold standard set, with a MAP of 62.9%. The automated ES achieved a 450% increase in trial screening efficiency. The findings on the gold standard set were confirmed by large-scale evaluation on the reference set of trial–patient matches. Discussion and conclusion By exploiting the text of trial criteria and the content of EHRs, we demonstrated that NLP-, IE-, and ML-based automated ES could successfully identify patients for clinical trials.
- Open Access
- 1 May 2014
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 21 (3) , 406-413
Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements
Objective To present a series of experiments: (1) to evaluate the impact of pre-annotation on the speed of manual annotation of clinical trial announ
- 1 May 2014
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 21 (3) , 406-413
Open AccessEvaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements
Objective To present a series of experiments: (1) to evaluate the impact of pre-annotation on the speed of manual annotation of clinical trial announ
- Open Access
- 1 December 2013
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 20 (e2) , e212-e220
Developing and evaluating an automated appendicitis risk stratification algorithm for pediatric patients in the emergency department
Objective To evaluate a proposed natural language processing (NLP) and machine-learning based automated method to risk stratify abdominal pain patient
- 1 December 2013
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 20 (e2) , e212-e220
Open AccessDeveloping and evaluating an automated appendicitis risk stratification algorithm for pediatric patients in the emergency department
Objective To evaluate a proposed natural language processing (NLP) and machine-learning based automated method to risk stratify abdominal pain patient
- Open Access
- 1 September 2013
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 20 (5) , 915-921
A sequence labeling approach to link medications and their attributes in clinical notes and clinical trial announcements for information extraction
Objective The goal of this work was to evaluate machine learning methods, binary classification and sequence labeling, for medication–attribute linkage detection in two clinical corpora. Data and methods We double annotated 3000 clinical trial announcements (CTA) and 1655 clinical notes (CN) for medication named entities and their attributes. A binary support vector machine (SVM) classification method with parsimonious feature sets, and a conditional random fields (CRF)-based multi-layered sequence labeling (MLSL) model were proposed to identify the linkages between the entities and their corresponding attributes. We evaluated the system's performance against the human-generated gold standard. Results The experiments showed that the two machine learning approaches performed statistically significantly better than the baseline rule-based approach. The binary SVM classification achieved 0.94 F-measure with individual tokens as features. The SVM model trained on a parsimonious feature set achieved 0.81 F-measure for CN and 0.87 for CTA. The CRF MLSL method achieved 0.80 F-measure on both corpora. Discussion and conclusions We compared the novel MLSL method with a binary classification and a rule-based method. The MLSL method performed statistically significantly better than the rule-based method. However, the SVM-based binary classification method was statistically significantly better than the MLSL method for both the CTA and CN corpora. Using parsimonious feature sets both the SVM-based binary classification and CRF-based MLSL methods achieved high performance in detecting medication name and attribute linkages in CTA and CN.
- 1 September 2013
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 20 (5) , 915-921
Open AccessA sequence labeling approach to link medications and their attributes in clinical notes and clinical trial announcements for information extraction
Objective The goal of this work was to evaluate machine learning methods, binary classification and sequence labeling, for medication–attribute linkage detection in two clinical corpora. Data and methods We double annotated 3000 clinical trial announcements (CTA) and 1655 clinical notes (CN) for medication named entities and their attributes. A binary support vector machine (SVM) classification method with parsimonious feature sets, and a conditional random fields (CRF)-based multi-layered sequence labeling (MLSL) model were proposed to identify the linkages between the entities and their corresponding attributes. We evaluated the system's performance against the human-generated gold standard. Results The experiments showed that the two machine learning approaches performed statistically significantly better than the baseline rule-based approach. The binary SVM classification achieved 0.94 F-measure with individual tokens as features. The SVM model trained on a parsimonious feature set achieved 0.81 F-measure for CN and 0.87 for CTA. The CRF MLSL method achieved 0.80 F-measure on both corpora. Discussion and conclusions We compared the novel MLSL method with a binary classification and a rule-based method. The MLSL method performed statistically significantly better than the rule-based method. However, the SVM-based binary classification method was statistically significantly better than the MLSL method for both the CTA and CN corpora. Using parsimonious feature sets both the SVM-based binary classification and CRF-based MLSL methods achieved high performance in detecting medication name and attribute linkages in CTA and CN.
- Open Access
- 24 April 2013
- Published by Springer Nature in BMC Medical Informatics and Decision Making
- Vol. 13 (1) , 53
Mining FDA drug labels for medical conditions
Background: Cincinnati Children’s Hospital Medical Center (CCHMC) has built the initial Natural Language Processing (NLP) component to extract medications with their corresponding medical conditions (Indications, Contraindications, Overdosage, and Adverse Reactions) as triples of medication-related information ([(1) drug name]-[(2) medical condition]-[(3) LOINC section header]) for an intelligent database system, in order to improve patient safety and the quality of health care. The Food and Drug Administration’s (FDA) drug labels are used to demonstrate the feasibility of building the triples as an intelligent database system task. Methods: This paper discusses a hybrid NLP system, called AutoMCExtractor, to collect medical conditions (including disease/disorder and sign/symptom) from drug labels published by the FDA. Altogether, 6,611 medical conditions in a manually-annotated gold standard were used for the system evaluation. The pre-processing step extracted the plain text from XML file and detected eight related LOINC sections (e.g. Adverse Reactions, Warnings and Precautions) for medical condition extraction. Conditional Random Fields (CRF) classifiers, trained on token, linguistic, and semantic features, were then used for medical condition extraction. Lastly, dictionary-based post-processing corrected boundary-detection errors of the CRF step. We evaluated the AutoMCExtractor on manually-annotated FDA drug labels and report the results on both token and span levels. Results: Precision, recall, and F-measure were 0.90, 0.81, and 0.85, respectively, for the span level exact match; for the token-level evaluation, precision, recall, and F-measure were 0.92, 0.73, and 0.82, respectively. Conclusions: The results demonstrate that (1) medical conditions can be extracted from FDA drug labels with high performance; and (2) it is feasible to develop a framework for an intelligent database system.
- 24 April 2013
- Published by Springer Nature in BMC Medical Informatics and Decision Making
- Vol. 13 (1) , 53
Open AccessMining FDA drug labels for medical conditions
Background: Cincinnati Children’s Hospital Medical Center (CCHMC) has built the initial Natural Language Processing (NLP) component to extract medications with their corresponding medical conditions (Indications, Contraindications, Overdosage, and Adverse Reactions) as triples of medication-related information ([(1) drug name]-[(2) medical condition]-[(3) LOINC section header]) for an intelligent database system, in order to improve patient safety and the quality of health care. The Food and Drug Administration’s (FDA) drug labels are used to demonstrate the feasibility of building the triples as an intelligent database system task. Methods: This paper discusses a hybrid NLP system, called AutoMCExtractor, to collect medical conditions (including disease/disorder and sign/symptom) from drug labels published by the FDA. Altogether, 6,611 medical conditions in a manually-annotated gold standard were used for the system evaluation. The pre-processing step extracted the plain text from XML file and detected eight related LOINC sections (e.g. Adverse Reactions, Warnings and Precautions) for medical condition extraction. Conditional Random Fields (CRF) classifiers, trained on token, linguistic, and semantic features, were then used for medical condition extraction. Lastly, dictionary-based post-processing corrected boundary-detection errors of the CRF step. We evaluated the AutoMCExtractor on manually-annotated FDA drug labels and report the results on both token and span levels. Results: Precision, recall, and F-measure were 0.90, 0.81, and 0.85, respectively, for the span level exact match; for the token-level evaluation, precision, recall, and F-measure were 0.92, 0.73, and 0.82, respectively. Conclusions: The results demonstrate that (1) medical conditions can be extracted from FDA drug labels with high performance; and (2) it is feasible to develop a framework for an intelligent database system.
- Open Access
- 2 April 2013
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 15 (4) , e73
Web 2.0-Based Crowdsourcing for High-Quality Gold Standard Development in Clinical Natural Language Processing
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet
- 2 April 2013
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 15 (4) , e73
Open AccessWeb 2.0-Based Crowdsourcing for High-Quality Gold Standard Development in Clinical Natural Language Processing
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet
- 1 November 2012
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 405-409
Web Page Recognition Algorithm Based on Link Analysis in Theme Search Engine
Web page recognition is a problem in the design of web crawler in theme search engine. This paper designs a web page recognition algorithm based on link analysis to solve this problem. The main idea of this algorithm is to get the relevant web page recognition model through a combination of link analysis and theme URL knowledge base, based on the idea of statistics and social network analysis. Through the experiment, the precision rate of this algorithm is over 93 percent, and the recall rate is up to 85.4 percent. So the experiment is significant, better than other web page recognition algorithm. Experimental results show the feasibility and effectiveness of this algorithm.
- 1 November 2012
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 405-409
Web Page Recognition Algorithm Based on Link Analysis in Theme Search Engine
Web page recognition is a problem in the design of web crawler in theme search engine. This paper designs a web page recognition algorithm based on link analysis to solve this problem. The main idea of this algorithm is to get the relevant web page recognition model through a combination of link analysis and theme URL knowledge base, based on the idea of statistics and social network analysis. Through the experiment, the precision rate of this algorithm is over 93 percent, and the recall rate is up to 85.4 percent. So the experiment is significant, better than other web page recognition algorithm. Experimental results show the feasibility and effectiveness of this algorithm.
- 1 September 2012
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 108
Pre-annotating Clinical Notes and Clinical Trial Announcements for Gold Standard Corpus Development: Evaluating the Impact on Annotation Speed and Potential Bias
In this study our aim was to present a series of experiments to evaluate the impact of pre-annotation: (1) on the speed of manual annotation of clinical notes and clinical trial announcements; and (2) test for potential bias if pre-annotation is utilized. The gold standard was 900 clinical trial announcements from clinicaltrials.gov website and 1655 clinical notes annotated for diagnoses, signs, symptoms, UMLS CUI and SNOMED CT codes. Two dictionary-based methods were used to pre-annotate the text. Annotation time savings ranged from 2.89% to 29.1% per entity. The pre-annotation did not reduce the IAA or annotator performance but reduced the time to annotate in every experiment. Dictionary-based pre-annotation is a feasible and practical method to reduce cost of annotation without introducing bias in the process.
- 1 September 2012
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 108
Pre-annotating Clinical Notes and Clinical Trial Announcements for Gold Standard Corpus Development: Evaluating the Impact on Annotation Speed and Potential Bias
In this study our aim was to present a series of experiments to evaluate the impact of pre-annotation: (1) on the speed of manual annotation of clinical notes and clinical trial announcements; and (2) test for potential bias if pre-annotation is utilized. The gold standard was 900 clinical trial announcements from clinicaltrials.gov website and 1655 clinical notes annotated for diagnoses, signs, symptoms, UMLS CUI and SNOMED CT codes. Two dictionary-based methods were used to pre-annotate the text. Annotation time savings ranged from 2.89% to 29.1% per entity. The pre-annotation did not reduce the IAA or annotator performance but reduced the time to annotate in every experiment. Dictionary-based pre-annotation is a feasible and practical method to reduce cost of annotation without introducing bias in the process.