Interpretable segmentation of medical free-text records based on word embeddings
Open Access
- 28 October 2021
- journal article
- research article
- Published by Springer Nature in Journal of Intelligent Information Systems
- Vol. 57 (3) , 447-465
Abstract
Medical free-text records store a lot of useful information that can be exploited in developing computer-supported medicine. However, extracting the knowledge from the unstructured text is difficult and depends on the language. In the paper, we apply Natural Language Processing methods to process raw medical texts in Polish and propose a new methodology for clustering of patients’ visits. We (1) extract medical terminology from a corpus of free-text clinical records, (2) annotate data with medical concepts, (3) compute vector representations of medical concepts and validate them on the proposed term analogy tasks, (4) compute visit representations as vectors, (5) introduce a new method for clustering of patients’ visits and (6) apply the method to a corpus of 100,000 visits. We use several approaches to visual exploration that facilitate interpretation of segments. With our method, we obtain stable and separated segments of visits which are positively validated against final medical diagnoses. In this paper we show how algorithm for segmentation of medical free-text records may be used to aid medical doctors. In addition to this, we share implementation of described methods with examples as open-source package .Keywords
All Related Versions
Funding Information
- Narodowe Centrum Badań i Rozwoju (POIR.01.01.01-00-0328/17)
- Narodowe Centrum Badań i Rozwoju (2017/27/B/ST6/0130t)
This publication has 23 references indexed in Scilit:
- SimCompass: Using Deep Learning Word Embeddings to Assess Cross-level Similarity2014
- Glove: Global Vectors for Word Representation2014
- Hybrid Text Segmentation for Hungarian Clinical RecordsLecture Notes in Computer Science, 2013
- A New, Fast and Accurate Algorithm for Hierarchical Clustering on Euclidean DistancesLecture Notes in Computer Science, 2013
- The Unified Medical Language System (UMLS): integrating biomedical terminologyNucleic Acids Research, 2004
- OPTICSACM SIGMOD Record, 1999
- BIRCHACM SIGMOD Record, 1996
- Silhouettes: A graphical aid to the interpretation and validation of cluster analysisJournal of Computational and Applied Mathematics, 1987
- Objective Criteria for the Evaluation of Clustering MethodsJournal of the American Statistical Association, 1971
- Hierarchical Grouping to Optimize an Objective FunctionJournal of the American Statistical Association, 1963