Comparing representations of long clinical texts for the task of patient note-identification
Preprint
Abstract
In this paper, we address the challenge of patient-note identification, which involves accurately matching an anonymized clinical note to its corresponding patient, represented by a set of related notes. This task has broad applications, including duplicate records detection and patient similarity analysis, which require robust patient-level representations. We explore various embedding methods, including Hierarchical Attention Networks (HAN), three-level Hierarchical Transformer Networks (HTN), LongFormer, and advanced BERT-based models, focusing on their ability to process mediumto-long clinical texts effectively. Additionally, we evaluate different pooling strategies (mean, max, and mean_max) for aggregating wordlevel embeddings into patient-level representations and we examine the impact of sliding windows on model performance. Our results indicate that BERT-based embeddings outperform traditional and hierarchical models, particularly in processing lengthy clinical notes and capturing nuanced patient representations. Among the pooling strategies, mean_max pooling consistently yields the best results, highlighting its ability to capture critical features from clinical notes. Furthermore, the reproduction of our results on both MIMIC dataset and Necker hospital data warehouse illustrates the generalizability of these approaches to real-world applications, emphasizing the importance of both embedding methods and aggregation strategies in optimizing patient-note identification and enhancing patient-level modeling.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Patient Identification Based on Deep Metric Learning for Preventing Human Errors in Follow-up X-Ray ExaminationsJournal of Digital Imaging, 2023
- Language models are an effective representation learning technique for electronic health record dataJournal of Biomedical Informatics, 2020
- Towards unstructured mortality prediction with free-text clinical notesJournal of Biomedical Informatics, 2020
- Deep representation learning of electronic health records to unlock patient stratification at scalenpj Digital Medicine, 2020
- BEHRT: Transformer for Electronic Health RecordsScientific Reports, 2020
- Representation Learning for Natural Language Processing2020
- Patient representation learning and interpretable evaluation using clinical notesJournal of Biomedical Informatics, 2018
- Scalable and accurate deep learning with electronic health recordsnpj Digital Medicine, 2018
- A clinician friendly data warehouse oriented toward narrative reports: Dr. WarehouseJournal of Biomedical Informatics, 2018
- Finding patients using similarity measures in a rare diseases-oriented clinical data warehouse: Dr. Warehouse and the needle in the needle stackJournal of Biomedical Informatics, 2017