Vincent Andrearczyk

Objectives: Robust automated classification of paramagnetic rim lesions (PRLs) and remyelinated lesions based on iron and myelin content in people with multiple sclerosis (pwMS). Materials and Methods: In this prospective study (2018–2022), three-dimensional (3D) brain magnetic resonance imaging (MRI) from 180 pwMS (mean age: 47 ± 14 years, 108 females) were acquired at 3T including fluid-attenuated inversion recovery, magnetization-prepared 2 rapid acquisition gradient echoes (MP2RAGE), and T2*-weighted segmented echo-planar imaging. Post-processed quantitative susceptibility mapping (QSM) and filtered phase unwrapped (PU) images were generated. Ground truth was established through expert manual rating. Performance was assessed using nested cross-validation (CV) including outer test sets. Three neural network configurations were evaluated: (1) PRL classification with QSM and MP2RAGE, (2) PRL classification with PU and MP2RAGE, and (3) multiple lesion phenotype (MLP) classification with QSM and MP2RAGE. Results: For PRL classification, our network achieves a top mean validation F1 score of 0.737 ± 0.027 in the MP2RAGE-QSM configuration trained on PRLs, and a best test performance of 0.709 ± 0.040 when trained on MLP. The MP2RAGE-PU configuration yielded validation and test F1 scores of 0.733 ± 0.021 and 0.662 ± 0.037, respectively. The QSM-based MLP classification achieved a macro F1 test score of 0.728 ± 0.012. Conclusion: Deep learning can automate the classification of PRLs and QSM lesion phenotypes with high accuracy, potentially facilitating the detection of PRLs (included in the new diagnostic criteria) and remyelinated lesions, guiding personalized treatment decisions in pwMS.

Objectives: To assess the performance of automatic rib fracture detection of an existing deep learning (DL) model, nnDetection, on a postmortem (PM) CT scan dataset, and to identify the main factors of domain shift between clinical and PM CT imaging. Background: Rib fracture detection and classification in forensic investigations is a time-consuming yet crucial task that can contribute to determine the cause of death. DL models are a promising tool, as recent research shows that radiologists using DL models can detect rib fractures in clinical CT scans at higher sensitivity and in shorter time. Methods: A dataset of 50 PMCT scans (24% women; age: mean 61, range 19 – 96 years) was retrospectively collected and annotated, and used to train a first instance of the model, nnDetPM. Another instance of the model, nnDetClin, was trained on data from another dataset, RibFrac, consisting of 660 clinical CT scans (36% women; age: mean 55, range 21 – 94 years). Results: On the PM testing set, nnDetPM achieved an average sensitivity of 70.2% and an average precision (at 0.1 intersection over union) of 78.1%, whereas nnDetClin fell far behind at 19.8% average sensitivity and 25.5% average precision, indicating a substantial impact of the domain shift from clinical to PM CT data. Further inspection of the results showed that the main factors of this domain shift were the position of the arms and the presence of medical ware in the image acquisition area of the PMCT scans. Conclusion: The performance of nnDetPM, with an average sensitivity of 70.2%, was notable and comparable to that of radiologists. However, more advanced techniques must be explored to decide if DL models can overcome the domain shift factors.

To date, several methods have been developed to explain deep learning algorithms for classification tasks. Recently, an adaptation of two of such methods has been proposed to generate instance-level explainable maps in a semantic segmentation scenario, such as multiple sclerosis (MS) lesion segmentation. In the mentioned work, a 3D U-Net was trained and tested for MS lesion segmentation, yielding an F1 score of 0.7006, and a positive predictive value (PPV) of 0.6265. The distribution of values in explainable maps exposed some differences between maps of true and false positive (TP/FP) examples. Inspired by those results, we explore in this paper the use of characteristics of lesion-specific saliency maps to refine segmentation and detection scores. We generate around 21000 maps from as many TP/FP lesions in a batch of 72 patients (training set) and 4868 from the 37 patients in the test set. 93 radiomic features extracted from the first set of maps were used to train a logistic regression model and classify TP versus FP. On the test set, F1 score and PPV were improved by a large margin when compared to the initial model, reaching 0.7450 and 0.7817, with 95% confidence intervals of [0.7358, 0.7547] and [0.7679, 0.7962], respectively. These results suggest that saliency maps can be used to refine prediction scores, boosting a model’s performances.

Filters are commonly used to enhance specific structures and patterns in images, such as vessels or peritumoral regions, to enable clinical insights beyond the visible image using radiomics. However, their lack of standardization restricts reproducibility and clinical translation of radiomics decision support tools. In this special report, teams of researchers who developed radiomics software participated in a three-phase study (September 2020 to December 2022) to establish a standardized set of filters. The first two phases focused on finding reference filtered images and reference feature values for commonly used convolutional filters: mean, Laplacian of Gaussian, Laws and Gabor kernels, separable and nonseparable wavelets (including decomposed forms), and Riesz transformations. In the first phase, 15 teams used digital phantoms to establish 33 reference filtered images of 36 filter configurations. In phase 2, 11 teams used a chest CT image to derive reference values for 323 of 396 features computed from filtered images using 22 filter and image processing configurations. Reference filtered images and feature values for Riesz transformations were not established. Reproducibility of standardized convolutional filters was validated on a public data set of multimodal imaging (CT, fluorodeoxyglucose PET, and T1-weighted MRI) in 51 patients with soft-tissue sarcoma. At validation, reproducibility of 486 features computed from filtered images using nine configurations × three imaging modalities was assessed using the lower bounds of 95% CIs of intraclass correlation coefficients. Out of 486 features, 458 were found to be reproducible across nine teams with lower bounds of 95% CIs of intraclass correlation coefficients greater than 0.75. In conclusion, eight filter types were standardized with reference filtered images and reference feature values for verifying and calibrating radiomics software packages. A web-based tool is available for compliance checking.

Interpreting the inner workings of deep learning models is crucial for establishing trust and ensuring model safety. Concept-based explanations have emerged as a superior approach that is more interpretable than feature attribution estimates such as pixel saliency. However, defining the concepts for the interpretability analysis biases the explanations by the user's expectations on the concepts. To address this, we propose a novel post-hoc unsupervised method that automatically uncovers the concepts learned by deep models during training. By decomposing the latent space of a layer in singular vectors and refining them by unsupervised clustering, we uncover concept vectors aligned with directions of high variance that are relevant to the model prediction, and that point to semantically distinct concepts. Our extensive experiments reveal that the majority of our concepts are readily understandable to humans, exhibit coherency, and bear relevance to the task at hand. Moreover, we showcase the practical utility of our method in dataset exploration, where our concept vectors successfully identify outlier training samples affected by various confounding factors. This novel exploration technique has remarkable versatility to data types and model architectures and it will facilitate the identification of biases and the discovery of sources of error within training data.

Adopting Convolutional Neural Networks (CNNs) in the daily routine of primary diagnosis requires not only near-perfect precision, but also a sufficient degree of generalization to data acquisition shifts and transparency. Existing CNN models act as black boxes, not ensuring to the physicians that important diagnostic features are used by the model. Building on top of successfully existing techniques such as multi-task learning, domain adversarial training and concept-based interpretability, this paper addresses the challenge of introducing diagnostic factors in the training objectives. Here we show that our architecture, by learning end-to-end an uncertainty-based weighting combination of multi-task and adversarial losses, is encouraged to focus on pathology features such as density and pleomorphism of nuclei, e.g. variations in size and appearance, while discarding misleading features such as staining differences. Our results on breast lymph node tissue show significantly improved generalization in the detection of tumorous tissue, with best average AUC 0.89 (0.01) against the baseline AUC 0.86 (0.005). By applying the interpretability technique of linearly probing intermediate representations, we also demonstrate that interpretable pathology features such as nuclei density are learned by the proposed CNN architecture, confirming the increased transparency of this model. This result is a starting point towards building interpretable multi-task architectures that are robust to data heterogeneity. Our code is available at https://github.com/maragraziani/multitask_adversarial