Tomohiro Wataya

The timely recognition of clinically important radiology findings is critical for patient safety, yet in Japan, no standardized multi-tiered system exists to classify the importance of radiology reports. This study aimed to propose a novel classification system for radiology reports based on the clinical importance of the findings called “Report Importance Category version 2 (RICv2)” and evaluate inter-radiologist agreement in its application. In RIC, reports are categorized into five categories; 1: “No/minor findings”, 2: “Maintain routine follow-up”, 3: “Closer follow-up required”, 4: “Action required within one month”, and 5: “Immediate action required”. Importance is defined not only by the severity of the patient, but also by the novelty and specificity of the findings. Five board-certified radiologists and five radiology residents reviewed and categorized 514 reports collected from Japan Medical Image Database (J-MID), and the inter-rater agreement was evaluated using unweighted and quadratic weighted Fleiss’ κ, and entropy analysis was applied to identify reports prone to disagreement. Among the 514 reports (408 CT, 106 MR), the distribution of final categories determined by majority consensus was RIC1 = 75 (14.6%), RIC2 = 327 (63.6%), RIC3 = 66 (12.8%), RIC4 = 34 (6.6%), and RIC5 = 12 (2.3%). Interobserver agreement was moderate, with unweighted Fleiss’ κ = 0.454, 0.478, and 0.449, for all radiologists, board-certified radiologists, and radiology residents, respectively, whereas substantial with weighted κ = 0.646, 0.669, and 0.662. Entropy analysis revealed that reports with multiple differentials or findings of uncertain clinical urgency tended to generate the greatest disagreement. RICv2 provides a structured, reproducible framework for classifying the importance of free-text radiology reports. Experience from other systems, such as Prostate Imaging Reporting and Data System (PI-RADS), suggests that refinement could further improve reproducibility, establishing RICv2 as a promising foundation for clinical and research applications.

Background: To investigate the performance of radiologists in characterizing and diagnosing hepatic lesions with and without the assistance of deep learning-based artificial intelligence (AI). Methods: This retrospective study included 83 nodules/masses from 69 patients who underwent dynamic contrast-enhanced CT of the liver. Image assessments were conducted by 20 radiologists. grouped according to their level of experience (10 senior and 10 junior). Each radiologist determined the probability of eight characteristics based on enhancement patterns and the diagnosis with and without AI attached to the SYNAPSE SAI viewer (FUJIFILM Corporation, Minato-ku, Japan). The reference standard for comparison was established as follows: final diagnoses were based on pathology for 39 lesions and expert imaging consensus for the remainder, while image characteristics for all lesions were determined by expert imaging consensus. Areas under the receiver operating characteristic curves (AUCs) were analyzed using the multireader multicase method. Results: Using AI significantly improved the overall AUCs for both the characterization and the diagnosis of liver lesions. Improvement was suggested for specific items, including the characterization of enhancement, nonperipheral washout, and delayed enhancement, and the diagnosis of hepatocellular carcinoma. The utilization of AI system also suggested potential improvements in the AUCs for image characterization in both the senior and junior groups. Conclusions: Using AI improved the radiologists’ performance in characterizing and diagnosing hepatic lesions. In terms of their capacity to assess imaging characteristics, improvements were observed regardless of their level of experience.

Purpose: Systems equipped with natural language (NLP) processing can reduce missed radiological findings by physicians, but the annotation costs are burden in the development. This study aimed to compare the effects of active learning (AL) algorithms in NLP for estimating the significance of head computed tomography (CT) reports using bidirectional encoder representations from transformers (BERT). Methods: A total of 3728 head CT reports annotated with five categories of importance were used and UTH-BERT was adopted as the pre-trained BERT model. We assumed that 64% (2385 reports) of the data were initially in the unlabeled data pool (UDP), while the labeled data set (LD) used to train the model was empty. Twenty-five reports were repeatedly selected from the UDP and added to the LD, based on seven metrices: random sampling (RS: control), four uncertainty sampling (US) methods (least confidence (LC), margin sampling (MS), ratio of confidence (RC), and entropy sampling (ES)), and two distance-based sampling (DS) methods (cosine distance (CD) and Euclidian distance (ED)). The transition of accuracy of the model was evaluated using the test dataset. Results: The accuracy of the models with US was significantly higher than RS when reports in LD were < 1800, whereas DS methods were significantly lower than RS. Among the US methods, MS and RC were even better than the others. With the US methods, the required labeled data decreased by 15.4–40.5%, and most efficient in RC. In addition, in the US methods, data for minor categories tended to be added to LD earlier than RS and DS. Conclusions: In the classification task for the importance of head CT reports, US methods, especially RC and MS can lead to the effective fine-tuning of BERT models and reduce the imbalance of categories. AL can contribute to other studies on larger datasets by providing effective annotation.

SUMMARY: Background: Artificial intelligence (AI) systems designed to detect abnormalities in abdominal computed tomography (CT) could reduce radiologists’ workload and improve diagnostic processes. However, development of such models has been hampered by the shortage of large expert-annotated datasets. Here, we used information from free-text radiology reports, rather than manual annotations, to develop a deep-learning-based pipeline for comprehensive detection of abdominal CT abnormalities. Methods: In this multicenter retrospective study, we developed a deep-learning-based pipeline to detect abnormalities in the liver, gallbladder, pancreas, spleen, and kidneys. Abdominal CT exams and related free-text reports obtained during routine clinical practice collected from three institutions were used for training and internal testing, while data collected from six institutions were used for external testing. A multi-organ segmentation model and an information extraction schema were used to extract specific organ images and disease information, CT images and radiology reports, respectively, which were used to train a multiple-instance learning model for anomaly detection. Its performance was evaluated using the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, and F1 score against radiologists’ ground-truth labels. Findings: We trained the model for each organ on images selected from 66,684 exams (39,255 patients) and tested it on 300 (295 patients) and 600 (596 patients) exams for internal and external validation, respectively. In the external test cohort, the overall AUC for detecting organ abnormalities was 0·886. Whereas models trained on human-annotated labels performed better with the same number of exams, those trained on larger datasets with labels auto-extracted via the information extraction schema significantly outperformed human-annotated label-derived models. Interpretation: Using disease information from routine clinical free-text radiology reports allows development of accurate anomaly detection models without requiring manual annotations. This approach is applicable to various anatomical sites and could streamline diagnostic processes. Funding: Japan Science and Technology Agency.

Purpose: To propose a five-point scale for radiology report importance called Report Importance Category (RIC) and to compare the performance of natural language processing (NLP) algorithms in assessing RIC using head computed tomography (CT) reports written in Japanese. Materials and methods: 3728 Japanese head CT reports performed at Osaka University Hospital in 2020 were included. RIC (category 0: no findings, category 1: minor findings, category 2: routine follow-up, category 3: careful follow-up, and category 4: examination or therapy) was established based not only on patient severity but also on the novelty of the information. The manual assessment of RIC for the reports was performed under the consensus of two out of four neuroradiologists. The performance of four NLP models for classifying RIC was compared using fivefold cross-validation: logistic regression, bidirectional long–short-term memory (BiLSTM), general bidirectional encoder representations of transformers (general BERT), and domain-specific BERT (BERT for medical domain). Results: The proportion of each RIC in the whole data set was 15.0%, 26.7%, 44.2%, 7.7%, and 6.4%, respectively. Domain-specific BERT showed the highest accuracy (0.8434 ± 0.0063) in assessing RIC and significantly higher AUC in categories 1 (0.9813 ± 0.0011), 2 (0.9492 ± 0.0045), 3 (0.9637 ± 0.0050), and 4 (0.9548 ± 0.0074) than the other models (p < .05). Analysis using layer-integrated gradients showed that the domain-specific BERT model could detect important words, such as disease names in reports. Conclusions: Domain-specific BERT has superiority over the other models in assessing our newly proposed criteria called RIC of head CT radiology reports. The accumulation of similar and further studies of has a potential to contribute to medical safety by preventing missed important findings by clinicians.

To assess whether transfer learning with a bidirectional encoder representations from transformers (BERT) model, pretrained on a clinical corpus, can perform sentence-level anatomic classification of free-text radiology reports, even for anatomic classes with few positive examples. This retrospective study included radiology reports of patients who underwent whole-body PET/CT imaging from December 2005 to December 2020. Each sentence in these reports (6272 sentences) was labeled by two annotators according to body part (“brain,” “head & neck,” “chest,” “abdomen,” “limbs,” “spine,” or “others”). The BERT-based transfer learning approach was compared with two baseline machine learning approaches: bidirectional long short-term memory (BiLSTM) and the count-based method. Area under the precision-recall curve (AUPRC) and area under the receiver operating characteristic curve (AUC) were computed for each approach, and AUCs were compared using the DeLong test. The BERT-based approach achieved a macro-averaged AUPRC of 0.88 for classification, outperforming the baselines. AUC results for BERT were significantly higher than those of BiLSTM for all classes and those of the count-based method for the “brain,” “chest,” “abdomen,” and “others” classes (P values < .025). AUPRC results for BERT were superior to those of baselines even for classes with few labeled training data (brain: BERT, 0.95, BiLSTM, 0.11, count based, 0.41; limbs: BERT, 0.74, BiLSTM, 0.28, count based, 0.46; spine: BERT, 0.82, BiLSTM, 0.53, count based, 0.69). The BERT-based transfer learning approach outperformed the BiLSTM and count-based approaches in sentence-level anatomic classification of free-text radiology reports, even for anatomic classes with few labeled training data. Keywords: Anatomy, Comparative Studies, Technology Assessment, Transfer Learning Supplemental material is available for this article. © RSNA, 2023

Objectives To compare the performance of radiologists in characterizing and diagnosing pulmonary nodules/masses with and without deep learning (DL)–based computer-aided diagnosis (CAD). Methods We studied a total of 101 nodules/masses detected on CT performed between January and March 2018 at Osaka University Hospital (malignancy: 55 cases). SYNAPSE SAI Viewer V1.4 was used to analyze the nodules/masses. In total, 15 independent radiologists were grouped (n = 5 each) according to their experience: L (< 3 years), M (3–5 years), and H (> 5 years). The likelihoods of 15 characteristics, such as cavitation and calcification, and the diagnosis (malignancy) were evaluated by each radiologist with and without CAD, and the assessment time was recorded. The AUCs compared with the reference standard set by two board-certified chest radiologists were analyzed following the multi-reader multi-case method. Furthermore, interobserver agreement was compared using intraclass correlation coefficients (ICCs). Results The AUCs for ill-defined boundary, irregular margin, irregular shape, calcification, pleural contact, and malignancy in all 15 radiologists, irregular margin and irregular shape in L and ill-defined boundary and irregular margin in M improved significantly (p < 0.05); no significant improvements were found in H. L showed the greatest increase in the AUC for malignancy (not significant). The ICCs improved in all groups and for nearly all items. The median assessment time was not prolonged by CAD. Conclusions DL-based CAD helps radiologists, particularly those with < 5 years of experience, to accurately characterize and diagnose pulmonary nodules/masses, and improves the reproducibility of findings among radiologists. Key Points • Deep learning–based computer-aided diagnosis improves the accuracy of characterizing nodules/masses and diagnosing malignancy, particularly by radiologists with < 5 years of experience. • Computer-aided diagnosis increases not only the accuracy but also the reproducibility of the findings across radiologists.

A pediatric case of cellulitis caused by Stenotrophomonas maltophilia after bone marrow transplantation was experienced. Pathologically, multiple microvascular thrombi and diffuse hemorrhage due to S. maltophilia cellulitis were observed in the subcutaneous tissue. This is the first report of pathological findings of such microvascular thrombi coexisting with hemorrhage in maltophilia infection.