Samuel G. Armato

Lung cancer is the leading cause of cancer death in men and women. The goal of reducing mortality from lung cancer requires improvements in many aspects of the care pathway, beginning with early detection. Advances in imaging technologies and their application in clinical practice can contribute to addressing these challenges. As part of the American Cancer Society National Lung Cancer Roundtable’s strategic plan, the roundtable’s Advanced Imaging Across the Lung Cancer Care Continuum Task Group identified four advanced imaging technology topics as priority areas for development. These include novel (1) imaging technologies, (2) quantitative imaging biomarkers, (3) systems for sharing image data and/or algorithms, and (4) imaging‐related artificial intelligence solutions. The authors provide background for each priority topic, describe key clinical problems along the lung cancer care pathway as use‐case scenarios, and comment on challenges that impede implementation and dissemination of these potential advances.

As COVID-19 transitions to an endemic phase, assessing lung involvement in affected patients remains vital for guiding treatment decisions and managing healthcare resources. Machine learning (ML) algorithms have proven invaluable for tasks such as image segmentation, registration, and computer-aided diagnosis, enabling the identification of patterns in complex medical imaging data that are not easily discernible through human analysis.¹ These capabilities enhance the accuracy and efficiency of radiologists and other medical professionals.²

PurposeThe Medical Imaging and Data Resource Center (MIDRC) mRALE Mastermind Grand Challenge fostered the development of artificial intelligence (AI) techniques for the automated assignment of mRALE (modified radiographic assessment of lung edema) scores to portable chest radiographs from patients known to have COVID-19.ApproachThe challenge utilized 2079 training cases obtained from the publicly available MIDRC data commons, with validation and test cases sampled from not-yet-public MIDRC cases that were inaccessible to challenge participants. The reference standard mRALE scores for the challenge cases were established by a pool of 22 radiologist annotators. Using the MedICI challenge platform, participants submitted their trained algorithms encapsulated in Docker containers. Algorithms were evaluated by the challenge organizers on 814 test cases through two performance assessment metrics: quadratic-weighted kappa and prediction probability concordance.ResultsNine AI algorithms were submitted to the challenge for assessment against the test set cases. The algorithm that demonstrated the highest agreement with the reference standard had a quadratic-weighted kappa of 0.885 and a prediction probability concordance of 0.875. Substantial variability in mRALE scores assigned by the annotators and output by the AI algorithms was observed.ConclusionsThe MIDRC mRALE Mastermind Grand Challenge revealed the potential of AI to assess COVID-19 severity from portable CXRs, demonstrating promising performance against the reference standard. The observed variability in mRALE scores highlights the challenges in standardizing severity assessment. These findings contribute to ongoing efforts to develop AI technologies for potential use in clinical practice and offer insights for the enhancement of COVID-19 severity assessment.

PurposeThis study aimed to investigate the impact of different model retraining schemes and data partitioning on model performance in the task of COVID-19 classification on standard chest radiographs (CXRs), in the context of model generalizability.ApproachTwo datasets from the same institution were used: Set A (9860 patients, collected from 02/20/2020 to 02/03/2021) and Set B (5893 patients, collected from 03/15/2020 to 01/01/2022). An original deep learning (DL) model trained and tested in the task of COVID-19 classification using the initial partition of Set A achieved an area under the curve (AUC) value of 0.76, whereas Set B yielded a significantly lower value of 0.67. To explore this discrepancy, four separate strategies were undertaken on the original model: (1) retrain using Set B, (2) fine-tune using Set B, (3) L2 regularization, and (4) repartition of the training set from Set A 200 times and report AUC values.ResultsThe model achieved the following AUC values (95% confidence interval) for the four methods: (1) 0.61 [0.56, 0.66]; (2) 0.70 [0.66, 0.73], both on Set B; (3) 0.76 [0.72, 0.79] on the initial test partition of Set A and 0.68 [0.66, 0.70] on Set B; and (4) 0.71±0.013 on repartitions of Set A. The lowest AUC value (0.66 [0.62, 0.69]) of the Set A repartitions was no longer significantly different from the initial 0.67 achieved on Set B.ConclusionsDifferent data repartitions of the same dataset used to train a DL model demonstrated significantly different performance values that helped explain the discrepancy between Set A and Set B and further demonstrated the limitations of model generalizability.

PurposeThe BRCA1-associated protein 1 (BAP1) gene is of great interest because somatic (BAP1) mutations are the most common alteration associated with pleural mesothelioma (PM). Further, germline mutation of the BAP1 gene has been linked to the development of PM. This study aimed to explore the potential of radiomics on computed tomography scans to identify somatic BAP1 gene mutations and assess the feasibility of radiomics in future research in identifying germline mutations.ApproachA cohort of 149 patients with PM and known somatic BAP1 mutation status was collected, and a previously published deep learning model was used to first automatically segment the tumor, followed by radiologist modifications. Image preprocessing was performed, and texture features were extracted from the segmented tumor regions. The top features were selected and used to train 18 separate machine learning models using leave-one-out cross-validation (LOOCV). The performance of the models in distinguishing between BAP1-mutated (BAP1+) and BAP1 wild-type (BAP1−) tumors was evaluated using the receiver operating characteristic area under the curve (ROC AUC).ResultsA decision tree classifier achieved the highest overall AUC value of 0.69 (95% confidence interval: 0.60 and 0.77). The features selected most frequently through the LOOCV were all second-order (gray-level co-occurrence or gray-level size zone matrices) and were extracted from images with an applied transformation.ConclusionsThis proof-of-concept work demonstrated the potential of radiomics to differentiate among BAP1+/− in patients with PM. Future work will extend these methods to the assessment of germline BAP1 mutation status through image analysis for improved patient prognostication.

The purpose of this study was to evaluate the impact of probability map threshold on pleural mesothelioma (PM) tumor delineations generated using a convolutional neural network (CNN). One hundred eighty-six CT scans from 48 PM patients were segmented by a VGG16/U-Net CNN. A radiologist modified the contours generated at a 0.5 probability threshold. Percent difference of tumor volume and overlap using the Dice Similarity Coefficient (DSC) were compared between the reference standard provided by the radiologist and CNN outputs for thresholds ranging from 0.001 to 0.9. CNN-derived contours consistently yielded smaller tumor volumes than radiologist contours. Reducing the probability threshold from 0.5 to 0.01 decreased the absolute percent volume difference, on average, from 42.93% to 26.60%. Median and mean DSC ranged from 0.57 to 0.59, with a peak at a threshold of 0.2; no distinct threshold was found for percent volume difference. The CNN exhibited deficiencies with specific disease presentations, such as severe pleural effusion or disease in the pleural fissure. No single output threshold in the CNN probability maps was optimal for both tumor volume and DSC. This study emphasized the importance of considering both figures of merit when evaluating deep learning-based tumor segmentations across probability thresholds. This work underscores the need to simultaneously assess tumor volume and spatial overlap when evaluating CNN performance. While automated segmentations may yield comparable tumor volumes to that of the reference standard, the spatial region delineated by the CNN at a specific threshold is equally important.

Malignant pleural mesothelioma (MPM) is the most common form of mesothelioma. To assess response to treatment, tumor measurements are acquired and evaluated based on a patient's longitudinal computed tomography (CT) scans. Tumor volume, however, is the more accurate metric for assessing tumor burden and response. Automated segmentation methods using deep learning can be employed to acquire volume, which otherwise is a tedious task performed manually. The deep learning-based tumor volume and contours can then be compared with a standard reference to assess the robustness of the automated segmentations. The purpose of this study was to evaluate the impact of probability map threshold on MPM tumor delineations generated using a convolutional neural network (CNN). Eighty-eight CT scans from 21 MPM patients were segmented by a VGG16/U-Net CNN. A radiologist modified the contours generated at a 0.5 probability threshold. Percent difference of tumor volume and overlap using the Dice Similarity Coefficient (DSC) were compared between the standard reference provided by the radiologist and CNN outputs for thresholds ranging from 0.001 to 0.9. CNN annotations consistently yielded smaller tumor volumes than radiologist contours. Reducing the probability threshold from 0.5 to 0.1 decreased the absolute percent volume difference, on average, from 43.96% to 24.18%. Median and mean DSC ranged from 0.58 to 0.60, with a peak at a threshold of 0.5; no distinct threshold was found for percent volume difference. No single output threshold in the CNN probability maps was optimal for both tumor volume and DSC. This work underscores the need to assess tumor volume and spatial overlap when evaluating CNN performance. While automated segmentations may yield comparable tumor volumes to that of the reference standard, the spatial region delineated by the CNN at a specific threshold is equally important.

PurposeThe purpose is to assess the performance of a pre-trained deep learning model in the task of classifying between coronavirus disease (COVID)-positive and COVID-negative patients from chest radiographs (CXRs) while considering various image acquisition parameters, clinical factors, and patient demographics.MethodsStandard and soft-tissue CXRs of 9860 patients comprised the “original dataset,” consisting of training and test sets and were used to train a DenseNet-121 architecture model to classify COVID-19 using three classification algorithms: standard, soft tissue, and a combination of both types of images via feature fusion. A larger more-current test set of 5893 patients (the “current test set”) was used to assess the performance of the pretrained model. The current test set contained a larger span of dates, incorporated different variants of the virus and included different immunization statuses. Model performance between the original and current test sets was evaluated using area under the receiver operating characteristic curve (ROC AUC) [95% CI].ResultsThe model achieved AUC values of 0.67 [0.65, 0.70] for cropped standard images, 0.65 [0.63, 0.67] for cropped soft-tissue images, and 0.67 [0.65, 0.69] for both types of cropped images. These were all significantly lower than the performance of the model on the original test set. Investigations regarding matching the acquisition dates between the test sets (i.e., controlling for virus variants), immunization status, disease severity, and age and sex distributions did not fully explain the discrepancy in performance.ConclusionsSeveral relevant factors were considered to determine whether differences existed in the test sets, including time period of image acquisition, vaccination status, and disease severity. The lower performance on the current test set may have occurred due to model overfitting and a lack of generalizability.

PurposeOur study aims to investigate the impact of preprocessing on magnetic resonance imaging (MRI) radiomic features extracted from the noncystic kidney parenchyma of patients with autosomal dominant polycystic kidney disease (ADPKD) in the task of classifying PKD1 versus PKD2 genotypes, which differ with regard to cyst burden and disease outcome.ApproachThe effect of preprocessing on radiomic features was investigated using a single T2-weighted fat saturated (T2W-FS) MRI scan from PKD1 and PKD2 subjects (29 kidneys in total) from the Consortium for Radiologic Imaging Studies of Polycystic Kidney Disease study. Radiomic feature reproducibility using the intraclass correlation coefficient (ICC) was computed across MRI normalizations (z-score, reference-tissue, and original image), gray-level discretization, and upsampling and downsampling pixel schemes. A second dataset for genotype classification from 136 subjects T2W-FS MRI images previously enrolled in the HALT Progression of Polycystic Kidney Disease study was matched for age, gender, and Mayo imaging classification class. Genotype classification was performed using a logistic regression classifier and radiomic features extracted from (1) the noncystic kidney parenchyma and (2) the entire kidney. The area under the receiver operating characteristic curve (AUC) was used to evaluate the classification performance across preprocessing methods.ResultsRadiomic features extracted from the noncystic kidney parenchyma were sensitive to preprocessing parameters, with varying reproducibility depending on the parameter. The percentage of features with good-to-excellent ICC scores ranged from 14% to 58%. AUC values ranged between 0.47 to 0.68 and 0.56 to 0.73 for the noncystic kidney parenchyma and entire kidney, respectively.ConclusionsReproducibility of radiomic features extracted from the noncystic kidney parenchyma was dependent on the preprocessing parameters used, and the effect on genotype classification was sensitive to preprocessing parameters. The results suggest that texture features may be indicative of genotype expression in ADPKD.