Mena Shenouda

PurposeThis study aimed to investigate the impact of different model retraining schemes and data partitioning on model performance in the task of COVID-19 classification on standard chest radiographs (CXRs), in the context of model generalizability.ApproachTwo datasets from the same institution were used: Set A (9860 patients, collected from 02/20/2020 to 02/03/2021) and Set B (5893 patients, collected from 03/15/2020 to 01/01/2022). An original deep learning (DL) model trained and tested in the task of COVID-19 classification using the initial partition of Set A achieved an area under the curve (AUC) value of 0.76, whereas Set B yielded a significantly lower value of 0.67. To explore this discrepancy, four separate strategies were undertaken on the original model: (1) retrain using Set B, (2) fine-tune using Set B, (3) L2 regularization, and (4) repartition of the training set from Set A 200 times and report AUC values.ResultsThe model achieved the following AUC values (95% confidence interval) for the four methods: (1) 0.61 [0.56, 0.66]; (2) 0.70 [0.66, 0.73], both on Set B; (3) 0.76 [0.72, 0.79] on the initial test partition of Set A and 0.68 [0.66, 0.70] on Set B; and (4) 0.71±0.013 on repartitions of Set A. The lowest AUC value (0.66 [0.62, 0.69]) of the Set A repartitions was no longer significantly different from the initial 0.67 achieved on Set B.ConclusionsDifferent data repartitions of the same dataset used to train a DL model demonstrated significantly different performance values that helped explain the discrepancy between Set A and Set B and further demonstrated the limitations of model generalizability.

PurposeThe BRCA1-associated protein 1 (BAP1) gene is of great interest because somatic (BAP1) mutations are the most common alteration associated with pleural mesothelioma (PM). Further, germline mutation of the BAP1 gene has been linked to the development of PM. This study aimed to explore the potential of radiomics on computed tomography scans to identify somatic BAP1 gene mutations and assess the feasibility of radiomics in future research in identifying germline mutations.ApproachA cohort of 149 patients with PM and known somatic BAP1 mutation status was collected, and a previously published deep learning model was used to first automatically segment the tumor, followed by radiologist modifications. Image preprocessing was performed, and texture features were extracted from the segmented tumor regions. The top features were selected and used to train 18 separate machine learning models using leave-one-out cross-validation (LOOCV). The performance of the models in distinguishing between BAP1-mutated (BAP1+) and BAP1 wild-type (BAP1−) tumors was evaluated using the receiver operating characteristic area under the curve (ROC AUC).ResultsA decision tree classifier achieved the highest overall AUC value of 0.69 (95% confidence interval: 0.60 and 0.77). The features selected most frequently through the LOOCV were all second-order (gray-level co-occurrence or gray-level size zone matrices) and were extracted from images with an applied transformation.ConclusionsThis proof-of-concept work demonstrated the potential of radiomics to differentiate among BAP1+/− in patients with PM. Future work will extend these methods to the assessment of germline BAP1 mutation status through image analysis for improved patient prognostication.

The purpose of this study was to evaluate the impact of probability map threshold on pleural mesothelioma (PM) tumor delineations generated using a convolutional neural network (CNN). One hundred eighty-six CT scans from 48 PM patients were segmented by a VGG16/U-Net CNN. A radiologist modified the contours generated at a 0.5 probability threshold. Percent difference of tumor volume and overlap using the Dice Similarity Coefficient (DSC) were compared between the reference standard provided by the radiologist and CNN outputs for thresholds ranging from 0.001 to 0.9. CNN-derived contours consistently yielded smaller tumor volumes than radiologist contours. Reducing the probability threshold from 0.5 to 0.01 decreased the absolute percent volume difference, on average, from 42.93% to 26.60%. Median and mean DSC ranged from 0.57 to 0.59, with a peak at a threshold of 0.2; no distinct threshold was found for percent volume difference. The CNN exhibited deficiencies with specific disease presentations, such as severe pleural effusion or disease in the pleural fissure. No single output threshold in the CNN probability maps was optimal for both tumor volume and DSC. This study emphasized the importance of considering both figures of merit when evaluating deep learning-based tumor segmentations across probability thresholds. This work underscores the need to simultaneously assess tumor volume and spatial overlap when evaluating CNN performance. While automated segmentations may yield comparable tumor volumes to that of the reference standard, the spatial region delineated by the CNN at a specific threshold is equally important.

Malignant pleural mesothelioma (MPM) is the most common form of mesothelioma. To assess response to treatment, tumor measurements are acquired and evaluated based on a patient's longitudinal computed tomography (CT) scans. Tumor volume, however, is the more accurate metric for assessing tumor burden and response. Automated segmentation methods using deep learning can be employed to acquire volume, which otherwise is a tedious task performed manually. The deep learning-based tumor volume and contours can then be compared with a standard reference to assess the robustness of the automated segmentations. The purpose of this study was to evaluate the impact of probability map threshold on MPM tumor delineations generated using a convolutional neural network (CNN). Eighty-eight CT scans from 21 MPM patients were segmented by a VGG16/U-Net CNN. A radiologist modified the contours generated at a 0.5 probability threshold. Percent difference of tumor volume and overlap using the Dice Similarity Coefficient (DSC) were compared between the standard reference provided by the radiologist and CNN outputs for thresholds ranging from 0.001 to 0.9. CNN annotations consistently yielded smaller tumor volumes than radiologist contours. Reducing the probability threshold from 0.5 to 0.1 decreased the absolute percent volume difference, on average, from 43.96% to 24.18%. Median and mean DSC ranged from 0.58 to 0.60, with a peak at a threshold of 0.5; no distinct threshold was found for percent volume difference. No single output threshold in the CNN probability maps was optimal for both tumor volume and DSC. This work underscores the need to assess tumor volume and spatial overlap when evaluating CNN performance. While automated segmentations may yield comparable tumor volumes to that of the reference standard, the spatial region delineated by the CNN at a specific threshold is equally important.

PurposeThe purpose is to assess the performance of a pre-trained deep learning model in the task of classifying between coronavirus disease (COVID)-positive and COVID-negative patients from chest radiographs (CXRs) while considering various image acquisition parameters, clinical factors, and patient demographics.MethodsStandard and soft-tissue CXRs of 9860 patients comprised the “original dataset,” consisting of training and test sets and were used to train a DenseNet-121 architecture model to classify COVID-19 using three classification algorithms: standard, soft tissue, and a combination of both types of images via feature fusion. A larger more-current test set of 5893 patients (the “current test set”) was used to assess the performance of the pretrained model. The current test set contained a larger span of dates, incorporated different variants of the virus and included different immunization statuses. Model performance between the original and current test sets was evaluated using area under the receiver operating characteristic curve (ROC AUC) [95% CI].ResultsThe model achieved AUC values of 0.67 [0.65, 0.70] for cropped standard images, 0.65 [0.63, 0.67] for cropped soft-tissue images, and 0.67 [0.65, 0.69] for both types of cropped images. These were all significantly lower than the performance of the model on the original test set. Investigations regarding matching the acquisition dates between the test sets (i.e., controlling for virus variants), immunization status, disease severity, and age and sex distributions did not fully explain the discrepancy in performance.ConclusionsSeveral relevant factors were considered to determine whether differences existed in the test sets, including time period of image acquisition, vaccination status, and disease severity. The lower performance on the current test set may have occurred due to model overfitting and a lack of generalizability.

Purpose: The goal of this study was to quantify the effects of iterative reconstruction on radiomics features of normal anatomic structures on head and neck computed tomography (CT) scans. Methods: Regions of interest (ROI) containing five different tissue types and an ROI containing only air were extracted from CT scans of the head and neck from 108 patients. Each scan was reconstructed using three different iDose⁴ reconstruction levels (2, 4, and 6) in addition to bone, thin slice (1-mm slice thickness), and thin-bone reconstructions. From each ROI in all reconstructions, 142 radiomic features were calculated. For each of the six ROIs, features were compared between combinations of iDose levels (2v4, 4v6, and 2v6) with a threshold of α = 0.05 after correcting for multiple comparisons (p < 0.00006). Features from iDose⁴-2 reconstructions were also compared to bone, thin slice, and thin-bone reconstructions. Spearman’s rank correlation coefficient, ρ, quantified the relative feature value agreement across iDose⁴ reconstructions. Results: When comparing radiomics features across the three iDose⁴ reconstruction levels, over half of all features reflected significant differences for all tissue types, while no features demonstrated significant differences when extracted from air ROIs. When assessing feature value agreement, at least 97% of features reflected excellent agreement (ρ > 0.9) when comparing the three iDose levels for all ROIs. When comparing iDose⁴-2 to bone, thin slice, and thin-bone reconstructions, more than half of all features demonstrated significant differences for all ROIs and 89 % of features reflected excellent agreement for all ROIs. Conclusion: Many radiomics features are dependent on the iterative reconstruction level, and the magnitude of this dependency is affected by the tissue from which features are extracted. For studies using images reconstructed using varying iDose⁴ reconstruction levels, features robust to these should be used.

In pulmonary hypertension and certain forms of congenital heart disease, ventricular pressure overload manifests at birth and is an obligate hemodynamic abnormality that stimulates myocardial fibrosis, which leads to ventricular dysfunction and poor clinical outcomes. Thus, an attractive strategy is to attenuate the myocardial fibrosis to help preserve ventricular function. Here, by analyzing RNA-sequencing databases and comparing the transcript and protein levels of fibrillar collagen in WT and global-knockout mice, we found that slit guidance ligand 3 (SLIT3) was present predominantly in fibrillar collagen–producing cells and that SLIT3 deficiency attenuated collagen production in the heart and other nonneuronal tissues. We then performed transverse aortic constriction or pulmonary artery banding to induce left and right ventricular pressure overload, respectively, in WT and knockout mice. We discovered that SLIT3 deficiency abrogated fibrotic and hypertrophic changes and promoted long-term ventricular function and overall survival in both left and right ventricular pressure overload. Furthermore, we found that SLIT3 stimulated fibroblast activity and fibrillar collagen production, which coincided with the transcription and nuclear localization of the mechanotransducer yes-associated protein 1. These results indicate that SLIT3 is important for regulating fibroblast activity and fibrillar collagen synthesis in an autocrine manner, making it a potential therapeutic target for fibrotic diseases, especially myocardial fibrosis and adverse remodeling induced by persistent afterload elevation.