Multilingual RECIST classification of radiology reports using supervised learning
Authors:
Open Access
- 14 June 2023
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Digital Health
- Vol. 5, 1195017
Abstract
Objectives: The objective of this study is the exploration of Artificial Intelligence and Natural Language Processing techniques to support the automatic assignment of the four Response Evaluation Criteria in Solid Tumors (RECIST) scales based on radiology reports. We also aim at evaluating how languages and institutional specificities of Swiss teaching hospitals are likely to affect the quality of the classification in French and German languages.Methods: In our approach, 7 machine learning methods were evaluated to establish a strong baseline. Then, robust models were built, fine-tuned according to the language (French and German), and compared with the expert annotation.Results: The best strategies yield average F1-scores of 90% and 86% respectively for the 2-classes (Progressive/Non-progressive) and the 4-classes (Progressive Disease, Stable Disease, Partial Response, Complete Response) RECIST classification tasks.Conclusions: These results are competitive with the manual labeling as measured by Matthew's correlation coefficient and Cohen's Kappa (79% and 76%). On this basis, we confirm the capacity of specific models to generalize on new unseen data and we assess the impact of using Pre-trained Language Models (PLMs) on the accuracy of the classifiers.Keywords
This publication has 22 references indexed in Scilit:
- Using and understanding cross-validation strategies. Perspectives on Saeb et al.GigaScience, 2017
- RECIST 1.1—Update and clarification: From the RECIST committeeEuropean Journal of Cancer, 2016
- Data and the clinical decision support loopJournal of the American Medical Informatics Association, 2016
- Classification of radiology reports for falls in an HIV study cohortJournal of the American Medical Informatics Association, 2015
- Interrater reliability: the kappa statisticBiochemia Medica, 2012
- An empirical assessment of validation practices for molecular classifiersBriefings in Bioinformatics, 2011
- New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1)European Journal of Cancer, 2009
- Advancing Biomedical Image Retrieval: Development and Analysis of a Test CollectionJournal of the American Medical Informatics Association, 2006
- Using lexical disambiguation and named-entity recognition to improve spelling correction in the electronic patient recordArtificial Intelligence in Medicine, 2003
- Comparison of the predicted and observed secondary structure of T4 phage lysozymeBiochimica et Biophysica Acta (BBA) - Protein Structure, 1975