A study of deep learning methods for de-identification of clinical notes in cross-institute settings
Open Access
- 5 December 2019
- journal article
- conference paper
- Published by Springer Nature in BMC Medical Informatics and Decision Making
- Vol. 19 (S5) , 1-9
Abstract
Background: De-identification is a critical technology to facilitate the use of unstructured clinical text while protecting patient privacy and confidentiality. The clinical natural language processing (NLP) community has invested great efforts in developing methods and corpora for de-identification of clinical notes. These annotated corpora are valuable resources for developing automated systems to de-identify clinical text at local hospitals. However, existing studies often utilized training and test data collected from the same institution. There are few studies to explore automated de-identification under cross-institute settings. The goal of this study is to examine deep learning-based de-identification methods at a cross-institute setting, identify the bottlenecks, and provide potential solutions. Methods: We created a de-identification corpus using a total 500 clinical notes from the University of Florida (UF) Health, developed deep learning-based de-identification models using 2014 i2b2/UTHealth corpus, and evaluated the performance using UF corpus. We compared five different word embeddings trained from the general English text, clinical text, and biomedical literature, explored lexical and linguistic features, and compared two strategies to customize the deep learning models using UF notes and resources. Results: Pre-trained word embeddings using a general English corpus achieved better performance than embeddings from de-identified clinical text and biomedical literature. The performance of deep learning models trained using only i2b2 corpus significantly dropped (strict and relax F1 scores dropped from 0.9547 and 0.9646 to 0.8568 and 0.8958) when applied to another corpus annotated at UF Health. Linguistic features could further improve the performance of de-identification in cross-institute settings. After customizing the models using UF notes and resource, the best model achieved the strict and relaxed F1 scores of 0.9288 and 0.9584, respectively. Conclusions: It is necessary to customize de-identification models using local clinical text and other resources when applied in cross-institute settings. Fine-tuning is a potential solution to re-use pre-trained parameters and reduce the training time to customize deep learning-based de-identification models trained using clinical corpus from a different institution.Keywords
This publication has 30 references indexed in Scilit:
- Automatic detection of protected health information from clinic narrativesJournal of Biomedical Informatics, 2015
- Automated systems for the de-identification of longitudinal clinical narratives: Overview of 2014 i2b2/UTHealth shared task Track 1Journal of Biomedical Informatics, 2015
- Evaluating the effects of machine pre-annotation and an interactive annotation interface on manual de-identification of clinical textJournal of Biomedical Informatics, 2014
- De-identification of health records using Anonym: Effectiveness and robustness across datasetsArtificial Intelligence in Medicine, 2014
- Evaluating current automatic de-identification methods with Veteran’s health administration clinical documentsBMC Medical Research Methodology, 2012
- Natural language processing: an introductionJournal of the American Medical Informatics Association, 2011
- A study of machine-learning-based approaches to extract clinical entities and their assertions from discharge summariesJournal of the American Medical Informatics Association, 2011
- Automatic de-identification of textual documents in the electronic health record: a review of recent researchBMC Medical Research Methodology, 2010
- Automated de-identification of free-text medical recordsBMC Medical Informatics and Decision Making, 2008
- Evaluating the State-of-the-Art in Automatic De-identificationJournal of the American Medical Informatics Association, 2007