Alison Deatsch
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Alison Deatsch has a total of 34 co-authors. Here are the top co-authors by number of shared publications:
- Robert Jeraj
Departments of Medical Physics and Human Oncology, University of Wisconsin, Madison, WI, USA; Department of Physics, University of Ljubljana, Ljubljana, Slovenia
8 shared publications - Zan Klanecek
Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia
6 shared publications - Brayden Schott
Medical Physics, University of Wisconsin-Madison, 1111 Highland Ave. #1005, Madison, Wisconsin, 53705, UNITED STATES
5 shared publications Andrej Studen
University of Ljubljana Faculty of Mathematics and Physics, Jadranska 19, Ljubljana, 1000, SLOVENIA
4 shared publicationsHilde Bosmans
Department of Radiology, UZ Gasthuisberg, 49 Herestraat, 3000 Leuven, Leuven, 3000, BELGIUM
4 shared publicationsKatja Jarm
Institute of Oncology Ljubljana, Zaloška cesta 2, Ljubljana, Ljubljana, 1000, SLOVENIA
4 shared publications
Latest Publications
- Open Access
- 24 October 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70 (21) , 215013
InfoOOD: information bottleneck optimization for post hoc medical image out-of-distribution detection
Objective. Deep learning models are prone to failure when inferring upon out-of-distribution (OOD) data, i.e. data whose features fundamentally differ from those in the training set. Existing OOD measures often lack sensitivity to the subtle image variations encountered within clinical settings. In this work, we investigate a post hoc, information-based approach to OOD detection—termed InfoOOD —which iteratively quantifies the amount of embedded feature information that can be shared between the training data and test data without degrading the model output. Approach. Abdominal CT images from patients with metastatic liver lesions were used. A 3D U-Net was trained to segment liver organs and lesions using N = 157 images. Physics-based artifacts—low dose, sparse view angles, and rings artifacts—were simulated on a separate set of N = 40 test images at three intensity magnitudes. Segmentation performance and the ability of the InfoOOD measure to detect the artifact-induced OOD data were evaluated. An additional N = 131 test images were used to assess the correlation between the InfoOOD measure and segmentation model performance metrics. In all evaluations, InfoOOD was compared with established embedded feature-based and reconstruction-based OOD detection methods. Results. Artifact simulation significantly degraded segmentation model performance across all artifact types and magnitudes ( ρ < 0.001), with model performance worsening as artifact magnitude increased. The InfoOOD measure consistently outperformed the embedded feature-based measures in detecting OOD data (e.g. AUC = 0.93 vs. AUC = 0.57 for the strong rings artifact) and surpassed the reconstruction-based measure across weak magnitude artifacts (e.g., AUC = 0.75 vs. AUC = 0.61 for the weak sparse view artifact). The InfoOOD measure also achieved stronger, negative correlations with segmentation performance metrics (e.g., ρ =−0.52 vs. ρ ⩾−0.11 for the lesion sensitivity metric). In both assessments, InfoOOD measure performance increased considerably with information bottleneck optimization iterations. Significance. This work introduces and validates a novel, highly sensitive, and clinically relevant information-theoretic approach for medical image OOD detection, supporting the safe deployment of deep learning models in clinical settings.
- 24 October 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70 (21) , 215013
Open AccessInfoOOD: information bottleneck optimization for post hoc medical image out-of-distribution detection
Objective. Deep learning models are prone to failure when inferring upon out-of-distribution (OOD) data, i.e. data whose features fundamentally differ from those in the training set. Existing OOD measures often lack sensitivity to the subtle image variations encountered within clinical settings. In this work, we investigate a post hoc, information-based approach to OOD detection—termed InfoOOD —which iteratively quantifies the amount of embedded feature information that can be shared between the training data and test data without degrading the model output. Approach. Abdominal CT images from patients with metastatic liver lesions were used. A 3D U-Net was trained to segment liver organs and lesions using N = 157 images. Physics-based artifacts—low dose, sparse view angles, and rings artifacts—were simulated on a separate set of N = 40 test images at three intensity magnitudes. Segmentation performance and the ability of the InfoOOD measure to detect the artifact-induced OOD data were evaluated. An additional N = 131 test images were used to assess the correlation between the InfoOOD measure and segmentation model performance metrics. In all evaluations, InfoOOD was compared with established embedded feature-based and reconstruction-based OOD detection methods. Results. Artifact simulation significantly degraded segmentation model performance across all artifact types and magnitudes ( ρ < 0.001), with model performance worsening as artifact magnitude increased. The InfoOOD measure consistently outperformed the embedded feature-based measures in detecting OOD data (e.g. AUC = 0.93 vs. AUC = 0.57 for the strong rings artifact) and surpassed the reconstruction-based measure across weak magnitude artifacts (e.g., AUC = 0.75 vs. AUC = 0.61 for the weak sparse view artifact). The InfoOOD measure also achieved stronger, negative correlations with segmentation performance metrics (e.g., ρ =−0.52 vs. ρ ⩾−0.11 for the lesion sensitivity metric). In both assessments, InfoOOD measure performance increased considerably with information bottleneck optimization iterations. Significance. This work introduces and validates a novel, highly sensitive, and clinically relevant information-theoretic approach for medical image OOD detection, supporting the safe deployment of deep learning models in clinical settings.
- Open Access
- 17 June 2025
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 20 (6) , e0325172
Prediction of future aging-related slow gait and its determinants with deep learning and logistic regression
Background: Identification of accelerated aging and its biomarkers can lead to more timely therapeutic interventions and decision-making. Therefore, we sought to predict aging-related slow gait, a known predictor of accelerated aging, and its determinants. Methods: We applied a deep learning neural network (NN) and compared it to conventional logistic regression (LR) analysis. We incorporated 1,363 participants from the Baltimore Longitudinal Study of Aging to predict current and future slow gait at 6-year and 10-year follow-up using two clinically-relevant cut-points. Results: Our NN achieved a maximum sensitivity (specificity) of 81.2% (87.9%), for a 10-year prediction with 0.8 m/s cut-point. We demonstrated the necessity of class balancing and found the NN to perform comparably to or in some cases, better than, LR which achieved a maximum sensitivity and specificity of 84.5% and 86.3%, respectively. Sobol index analysis identified the strongest determinants to be age, BMI, sleep, and grip strength. Conclusions: The novel use of a NN for this purpose, and successful benchmarking against conventional techniques, justifies further exploration and expansion of this model.
- 17 June 2025
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 20 (6) , e0325172
Open AccessPrediction of future aging-related slow gait and its determinants with deep learning and logistic regression
Background: Identification of accelerated aging and its biomarkers can lead to more timely therapeutic interventions and decision-making. Therefore, we sought to predict aging-related slow gait, a known predictor of accelerated aging, and its determinants. Methods: We applied a deep learning neural network (NN) and compared it to conventional logistic regression (LR) analysis. We incorporated 1,363 participants from the Baltimore Longitudinal Study of Aging to predict current and future slow gait at 6-year and 10-year follow-up using two clinically-relevant cut-points. Results: Our NN achieved a maximum sensitivity (specificity) of 81.2% (87.9%), for a 10-year prediction with 0.8 m/s cut-point. We demonstrated the necessity of class balancing and found the NN to perform comparably to or in some cases, better than, LR which achieved a maximum sensitivity and specificity of 84.5% and 86.3%, respectively. Sobol index analysis identified the strongest determinants to be age, BMI, sleep, and grip strength. Conclusions: The novel use of a NN for this purpose, and successful benchmarking against conventional techniques, justifies further exploration and expansion of this model.
- Open Access
- 17 April 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70 (8) , 085014
Sensitivity of a deep-learning-based breast cancer risk prediction model
Objective. When it comes to the implementation of deep-learning based breast cancer risk (BCR) prediction models in clinical settings, it is important to be aware that these models could be sensitive to various factors, especially those arising from the acquisition process. In this work, we investigated how sensitive the state-of-the-art BCR prediction model is to realistic image alterations that can occur as a result of different positioning during the acquisition process. Approach. 5076 mammograms (1269 exams, 650 participants) from the Slovenian and Belgium (University Hospital Leuven) Breast Cancer Screening Programs were collected. The Original MIRAI model was used for 1–5 year BCR estimation. First, BCR was predicted for the original mammograms, which were not changed. Then, a series of different image alteration techniques was performed, such as swapping left and right breasts, removing tissue below the inframammary fold, translations, cropping, rotations, registration and pectoral muscle removal. In addition, a subset of 81 exams, where at least one of the mammograms had to be retaken due to inadequate image quality, served as an approximation of a test–retest experiment. Bland–Altman plots were used to determine prediction bias and 95% limits of agreement (LOA). Additionally, the mean absolute difference in BCR (Mean AD) was calculated. The impact on the overall discrimination performance was evaluated with the AUC. Results. Swapping left and right breasts had no impact on the predicted BCR. The removal of skin tissue below the inframammary fold had minimal impact on the predicted BCR (1–5 year LOA: [−0.02, 0.01]). The model was sensitive to translation, rotation, registration, and cropping, where LOAs of up to ±0.1 were observed. Partial pectoral muscle removal did not have a major impact on predicted BCR, while complete removal of pectoral muscle introduced substantial prediction bias and LOAs (1 year LOA: [−0.07, 0.04], 5 year LOA: [−0.06, 0.03]). The approximation of a real test–retest experiment resulted in LOAs similar to those of simulated image alterations. None of the alterations impacted the overall BCR discrimination performance; the initial 1 year AUC (0.90 [0.88, 0.92]) and 5 year AUC (0.77 [0.75, 0.80]) remained unchanged. Significance. While tested image alterations do not impact overall BCR discrimination performance, substantial changes in predicted 1–5 year BCR can occur on an individual basis.
- 17 April 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70 (8) , 085014
Open AccessSensitivity of a deep-learning-based breast cancer risk prediction model
Objective. When it comes to the implementation of deep-learning based breast cancer risk (BCR) prediction models in clinical settings, it is important to be aware that these models could be sensitive to various factors, especially those arising from the acquisition process. In this work, we investigated how sensitive the state-of-the-art BCR prediction model is to realistic image alterations that can occur as a result of different positioning during the acquisition process. Approach. 5076 mammograms (1269 exams, 650 participants) from the Slovenian and Belgium (University Hospital Leuven) Breast Cancer Screening Programs were collected. The Original MIRAI model was used for 1–5 year BCR estimation. First, BCR was predicted for the original mammograms, which were not changed. Then, a series of different image alteration techniques was performed, such as swapping left and right breasts, removing tissue below the inframammary fold, translations, cropping, rotations, registration and pectoral muscle removal. In addition, a subset of 81 exams, where at least one of the mammograms had to be retaken due to inadequate image quality, served as an approximation of a test–retest experiment. Bland–Altman plots were used to determine prediction bias and 95% limits of agreement (LOA). Additionally, the mean absolute difference in BCR (Mean AD) was calculated. The impact on the overall discrimination performance was evaluated with the AUC. Results. Swapping left and right breasts had no impact on the predicted BCR. The removal of skin tissue below the inframammary fold had minimal impact on the predicted BCR (1–5 year LOA: [−0.02, 0.01]). The model was sensitive to translation, rotation, registration, and cropping, where LOAs of up to ±0.1 were observed. Partial pectoral muscle removal did not have a major impact on predicted BCR, while complete removal of pectoral muscle introduced substantial prediction bias and LOAs (1 year LOA: [−0.07, 0.04], 5 year LOA: [−0.06, 0.03]). The approximation of a real test–retest experiment resulted in LOAs similar to those of simulated image alterations. None of the alterations impacted the overall BCR discrimination performance; the initial 1 year AUC (0.90 [0.88, 0.92]) and 5 year AUC (0.77 [0.75, 0.80]) remained unchanged. Significance. While tested image alterations do not impact overall BCR discrimination performance, substantial changes in predicted 1–5 year BCR can occur on an individual basis.
- Open Access
- 18 February 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70 (5) , 055006
Impact of pectoral muscle removal on deep-learning-based breast cancer risk prediction
Objective. State-of-the-art breast cancer risk (BCR) prediction models have been originally trained on mammograms with pectoral muscle (PM) included. This study investigated whether excluding PM during training/fine-tuning improves the model’s BCR discrimination performance, calibration, and robustness. Approach. First, the Original deep learning model (MIRAI), trained on the US (Massachusetts General Hospital) data, was validated, and the relative contribution of PM to BCR predictions was evaluated using saliency maps. Additionally, 23 792 mammograms from the Slovenian screening program were collected and two datasets were created, with and without screening positive exams. The original MIRAI was then fine-tuned on the training/fine-tuning set of Slovenian mammograms with and without PM, creating Fine-tuned MIRAI models. In total, four models (Original MIRAI with PM, Original MIRAI without PM, Fine-tuned MIRAI with PM, Fine-tuned MIRAI without PM) were compared on a test set in terms of discrimination performance for 1–5 Year BCR (evaluating area under the curve), calibration performance (measured with expected calibration error—ECE) and robustness to incremental PM removals/additions, and to incremental breast tissue removals. Results. The relative contribution of PM to the BCR prediction on the Original MIRAI model was low (∼5%); however, there were significant outliers where the relative contribution was more than 50%. The removal of PM did not impact the 1–5 Year BCR discrimination performance of the Original MIRAI (with screening positive exams: 0.77–0.91, without screening positive exams: 0.64–0.67). Fine-tuned MIRAI on mammograms with PM removed achieved significantly higher 1-5 Year BCR discrimination performance (with screening positive exams: 0.82–0.93, without screening positive exams: 0.71–0.79). After recalibration, all models had similar ECE (with screening positive exams: 0.04–0.05, without screening positive exams: 0.02–0.03). Significance. Improved BCR discrimination performance can be achieved when the model is trained/fine-tuned on mammograms with PM removed.
- 18 February 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70 (5) , 055006
Open AccessImpact of pectoral muscle removal on deep-learning-based breast cancer risk prediction
Objective. State-of-the-art breast cancer risk (BCR) prediction models have been originally trained on mammograms with pectoral muscle (PM) included. This study investigated whether excluding PM during training/fine-tuning improves the model’s BCR discrimination performance, calibration, and robustness. Approach. First, the Original deep learning model (MIRAI), trained on the US (Massachusetts General Hospital) data, was validated, and the relative contribution of PM to BCR predictions was evaluated using saliency maps. Additionally, 23 792 mammograms from the Slovenian screening program were collected and two datasets were created, with and without screening positive exams. The original MIRAI was then fine-tuned on the training/fine-tuning set of Slovenian mammograms with and without PM, creating Fine-tuned MIRAI models. In total, four models (Original MIRAI with PM, Original MIRAI without PM, Fine-tuned MIRAI with PM, Fine-tuned MIRAI without PM) were compared on a test set in terms of discrimination performance for 1–5 Year BCR (evaluating area under the curve), calibration performance (measured with expected calibration error—ECE) and robustness to incremental PM removals/additions, and to incremental breast tissue removals. Results. The relative contribution of PM to the BCR prediction on the Original MIRAI model was low (∼5%); however, there were significant outliers where the relative contribution was more than 50%. The removal of PM did not impact the 1–5 Year BCR discrimination performance of the Original MIRAI (with screening positive exams: 0.77–0.91, without screening positive exams: 0.64–0.67). Fine-tuned MIRAI on mammograms with PM removed achieved significantly higher 1-5 Year BCR discrimination performance (with screening positive exams: 0.82–0.93, without screening positive exams: 0.71–0.79). After recalibration, all models had similar ECE (with screening positive exams: 0.04–0.05, without screening positive exams: 0.02–0.03). Significance. Improved BCR discrimination performance can be achieved when the model is trained/fine-tuned on mammograms with PM removed.
- Open Access
- 24 December 2024
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70 (1) , 015001
Longitudinal interpretability of deep learning based breast cancer risk prediction
: Objective. Deep-learning-based models have achieved state-of-the-art breast cancer risk (BCR) prediction performance. However, these models are highly complex, and the underlying mechanisms of BCR prediction are not fully understood. Key questions include whether these models can detect breast morphologic changes that lead to cancer. These findings would boost confidence in utilizing BCR models in practice and provide clinicians with new perspectives. In this work, we aimed to determine when oncogenic processes in the breast provide sufficient signal for the models to detect these changes. Approach. In total, 1210 screening mammograms were collected for patients screened at different times before the cancer was screen-detected and 2400 mammograms for patients with at least ten years of follow-up. MIRAI, a BCR risk prediction model, was used to estimate the BCR. Attribution heterogeneity was defined as the relative difference between the attributions obtained from the right and left breasts using one of the eight interpretability techniques. Model reliance on the side of the breast with cancer was quantified with AUC. The Mann–Whitney U test was used to check for significant differences in median absolute Attribution Heterogeneity between cancer patients and healthy individuals. Results. All tested attribution methods showed a similar longitudinal trend, where the model reliance on the side of the breast with cancer was the highest for the 0–1 years-to-cancer interval (AUC = 0.85–0.95), dropped for the 1–3 years-to-cancer interval (AUC = 0.64–0.71), and remained above the threshold for random performance for the 3–5 years-to-cancer interval (AUC = 0.51–0.58). For all eight attribution methods, the median values of absolute attribution heterogeneity were significantly larger for patients diagnosed with cancer at one point (p < 0.01). Significance. Interpretability of BCR prediction has revealed that long-term predictions (beyond three years) are most likely based on typical breast characteristics, such as breast density; for mid-term predictions (one to three years), the model appears to detect early signs of tumor development, while for short-term predictions (up to a year), the BCR model essentially functions as a breast cancer detection model.
- 24 December 2024
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70 (1) , 015001
Open AccessLongitudinal interpretability of deep learning based breast cancer risk prediction
: Objective. Deep-learning-based models have achieved state-of-the-art breast cancer risk (BCR) prediction performance. However, these models are highly complex, and the underlying mechanisms of BCR prediction are not fully understood. Key questions include whether these models can detect breast morphologic changes that lead to cancer. These findings would boost confidence in utilizing BCR models in practice and provide clinicians with new perspectives. In this work, we aimed to determine when oncogenic processes in the breast provide sufficient signal for the models to detect these changes. Approach. In total, 1210 screening mammograms were collected for patients screened at different times before the cancer was screen-detected and 2400 mammograms for patients with at least ten years of follow-up. MIRAI, a BCR risk prediction model, was used to estimate the BCR. Attribution heterogeneity was defined as the relative difference between the attributions obtained from the right and left breasts using one of the eight interpretability techniques. Model reliance on the side of the breast with cancer was quantified with AUC. The Mann–Whitney U test was used to check for significant differences in median absolute Attribution Heterogeneity between cancer patients and healthy individuals. Results. All tested attribution methods showed a similar longitudinal trend, where the model reliance on the side of the breast with cancer was the highest for the 0–1 years-to-cancer interval (AUC = 0.85–0.95), dropped for the 1–3 years-to-cancer interval (AUC = 0.64–0.71), and remained above the threshold for random performance for the 3–5 years-to-cancer interval (AUC = 0.51–0.58). For all eight attribution methods, the median values of absolute attribution heterogeneity were significantly larger for patients diagnosed with cancer at one point (p < 0.01). Significance. Interpretability of BCR prediction has revealed that long-term predictions (beyond three years) are most likely based on typical breast characteristics, such as breast density; for mid-term predictions (one to three years), the model appears to detect early signs of tumor development, while for short-term predictions (up to a year), the BCR model essentially functions as a breast cancer detection model.
- Open Access
- 19 July 2024
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 69 (15) , 155015
Uncertainty quantification via localized gradients for deep learning-based medical image assessments
Objective. Deep learning models that aid in medical image assessment tasks must be both accurate and reliable to be deployed within clinical settings. While deep learning models have been shown to be highly accurate across a variety of tasks, measures that indicate the reliability of these models are less established. Increasingly, uncertainty quantification (UQ) methods are being introduced to inform users on the reliability of model outputs. However, most existing methods cannot be augmented to previously validated models because they are not post hoc, and they change a model’s output. In this work, we overcome these limitations by introducing a novel post hoc UQ method, termed Local Gradients UQ, and demonstrate its utility for deep learning-based metastatic disease delineation. Approach. This method leverages a trained model’s localized gradient space to assess sensitivities to trained model parameters. We compared the Local Gradients UQ method to non-gradient measures defined using model probability outputs. The performance of each uncertainty measure was assessed in four clinically relevant experiments: (1) response to artificially degraded image quality, (2) comparison between matched high- and low-quality clinical images, (3) false positive (FP) filtering, and (4) correspondence with physician-rated disease likelihood. Main results. (1) Response to artificially degraded image quality was enhanced by the Local Gradients UQ method, where the median percent difference between matching lesions in non-degraded and most degraded images was consistently higher for the Local Gradients uncertainty measure than the non-gradient uncertainty measures (e.g. 62.35% vs. 2.16% for additive Gaussian noise). (2) The Local Gradients UQ measure responded better to high- and low-quality clinical images (p < 0.05 vs p > 0.1 for both non-gradient uncertainty measures). (3) FP filtering performance was enhanced by the Local Gradients UQ method when compared to the non-gradient methods, increasing the area under the receiver operating characteristic curve (ROC AUC) by 20.1% and decreasing the false positive rate by 26%. (4) The Local Gradients UQ method also showed more favorable correspondence with physician-rated likelihood for malignant lesions by increasing ROC AUC for correspondence with physician-rated disease likelihood by 16.2%. Significance. In summary, this work introduces and validates a novel gradient-based UQ method for deep learning-based medical image assessments to enhance user trust when using deployed clinical models.
- 19 July 2024
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 69 (15) , 155015
Open AccessUncertainty quantification via localized gradients for deep learning-based medical image assessments
Objective. Deep learning models that aid in medical image assessment tasks must be both accurate and reliable to be deployed within clinical settings. While deep learning models have been shown to be highly accurate across a variety of tasks, measures that indicate the reliability of these models are less established. Increasingly, uncertainty quantification (UQ) methods are being introduced to inform users on the reliability of model outputs. However, most existing methods cannot be augmented to previously validated models because they are not post hoc, and they change a model’s output. In this work, we overcome these limitations by introducing a novel post hoc UQ method, termed Local Gradients UQ, and demonstrate its utility for deep learning-based metastatic disease delineation. Approach. This method leverages a trained model’s localized gradient space to assess sensitivities to trained model parameters. We compared the Local Gradients UQ method to non-gradient measures defined using model probability outputs. The performance of each uncertainty measure was assessed in four clinically relevant experiments: (1) response to artificially degraded image quality, (2) comparison between matched high- and low-quality clinical images, (3) false positive (FP) filtering, and (4) correspondence with physician-rated disease likelihood. Main results. (1) Response to artificially degraded image quality was enhanced by the Local Gradients UQ method, where the median percent difference between matching lesions in non-degraded and most degraded images was consistently higher for the Local Gradients uncertainty measure than the non-gradient uncertainty measures (e.g. 62.35% vs. 2.16% for additive Gaussian noise). (2) The Local Gradients UQ measure responded better to high- and low-quality clinical images (p < 0.05 vs p > 0.1 for both non-gradient uncertainty measures). (3) FP filtering performance was enhanced by the Local Gradients UQ method when compared to the non-gradient methods, increasing the area under the receiver operating characteristic curve (ROC AUC) by 20.1% and decreasing the false positive rate by 26%. (4) The Local Gradients UQ method also showed more favorable correspondence with physician-rated likelihood for malignant lesions by increasing ROC AUC for correspondence with physician-rated disease likelihood by 16.2%. Significance. In summary, this work introduces and validates a novel gradient-based UQ method for deep learning-based medical image assessments to enhance user trust when using deployed clinical models.
- Open Access
- 3 April 2024
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 69 (8) , 085012
An automated methodology for whole-body, multimodality tracking of individual cancer lesions
Objective . Manual analysis of individual cancer lesions to assess disease response is clinically impractical and requires automated lesion tracking methodologies. However, no methodology has been developed for whole-body individual lesion tracking, across an arbitrary number of scans, and acquired with various imaging modalities. Approach . This study introduces a lesion tracking methodology and benchmarked it using 23 68 Ga-DOTATATE PET/CT and PET/MR images of eight neuroendocrine tumor patients. The methodology consists of six steps: (1) alignment of multiple scans via image registration, (2) body-part labeling, (3) automatic lesion-wise dilation, (4) clustering of lesions based on local lesion shape metrics, (5) assignment of lesion tracks, and (6) output of a lesion graph. Registration performance was evaluated via landmark distance, lesion matching accuracy was evaluated between each image pair, and lesion tracking accuracy was evaluated via identical track ratio. Sensitivity studies were performed to evaluate the impact of lesion dilation (fixed versus automatic dilation), anatomic location, image modalities (inter- versus intra-modality), registration mode (direct versus indirect registration), and track size (number of time-points and lesions) on lesion matching and tracking performance. Main results . Manual contouring yielded 956 lesions, 1570 lesion-matching decisions, and 493 lesion tracks. The median residual registration error was 2.5 mm. The automatic lesion dilation led to 0.90 overall lesion matching accuracy, and an 88% identical track ratio. The methodology is robust regarding anatomic locations, image modalities, and registration modes. The number of scans had a moderate negative impact on the identical track ratio (94% for 2 scans, 91% for 3 scans, and 81% for 4 scans). The number of lesions substantially impacted the identical track ratio (93% for 2 nodes versus 54% for ≥5 nodes). Significance . The developed methodology resulted in high lesion-matching accuracy and enables automated lesion tracking in PET/CT and PET/MR.
- 3 April 2024
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 69 (8) , 085012
Open AccessAn automated methodology for whole-body, multimodality tracking of individual cancer lesions
Objective . Manual analysis of individual cancer lesions to assess disease response is clinically impractical and requires automated lesion tracking methodologies. However, no methodology has been developed for whole-body individual lesion tracking, across an arbitrary number of scans, and acquired with various imaging modalities. Approach . This study introduces a lesion tracking methodology and benchmarked it using 23 68 Ga-DOTATATE PET/CT and PET/MR images of eight neuroendocrine tumor patients. The methodology consists of six steps: (1) alignment of multiple scans via image registration, (2) body-part labeling, (3) automatic lesion-wise dilation, (4) clustering of lesions based on local lesion shape metrics, (5) assignment of lesion tracks, and (6) output of a lesion graph. Registration performance was evaluated via landmark distance, lesion matching accuracy was evaluated between each image pair, and lesion tracking accuracy was evaluated via identical track ratio. Sensitivity studies were performed to evaluate the impact of lesion dilation (fixed versus automatic dilation), anatomic location, image modalities (inter- versus intra-modality), registration mode (direct versus indirect registration), and track size (number of time-points and lesions) on lesion matching and tracking performance. Main results . Manual contouring yielded 956 lesions, 1570 lesion-matching decisions, and 493 lesion tracks. The median residual registration error was 2.5 mm. The automatic lesion dilation led to 0.90 overall lesion matching accuracy, and an 88% identical track ratio. The methodology is robust regarding anatomic locations, image modalities, and registration modes. The number of scans had a moderate negative impact on the identical track ratio (94% for 2 scans, 91% for 3 scans, and 81% for 4 scans). The number of lesions substantially impacted the identical track ratio (93% for 2 nodes versus 54% for ≥5 nodes). Significance . The developed methodology resulted in high lesion-matching accuracy and enables automated lesion tracking in PET/CT and PET/MR.
- Open Access
- 22 May 2023
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 68 (11) , 115007
Uncertainty estimation for deep learning-based pectoral muscle segmentation via Monte Carlo dropout
Objective . Deep Learning models are often susceptible to failures after deployment. Knowing when your model is producing inadequate predictions is crucial. In this work, we investigate the utility of Monte Carlo (MC) dropout and the efficacy of the proposed uncertainty metric (UM) for flagging of unacceptable pectoral muscle segmentations in mammograms. Approach . Segmentation of pectoral muscle was performed with modified ResNet18 convolutional neural network. MC dropout layers were kept unlocked at inference time. For each mammogram, 50 pectoral muscle segmentations were generated. The mean was used to produce the final segmentation and the standard deviation was applied for the estimation of uncertainty. From each pectoral muscle uncertainty map, the overall UM was calculated. To validate the UM, a correlation between the dice similarity coefficient (DSC) and UM was used. The UM was first validated in a training set (200 mammograms) and finally tested in an independent dataset (300 mammograms). ROC-AUC analysis was performed to test the discriminatory power of the proposed UM for flagging unacceptable segmentations. Main results . The introduction of dropout layers in the model improved segmentation performance (DSC = 0.95 ± 0.07 versus DSC = 0.93 ± 0.10). Strong anti-correlation ( r = −0.76, p < 0.001) between the proposed UM and DSC was observed. A high AUC of 0.98 (97% specificity at 100% sensitivity) was obtained for the discrimination of unacceptable segmentations. Qualitative inspection by the radiologist revealed that images with high UM are difficult to segment. Significance . The use of MC dropout at inference time in combination with the proposed UM enables flagging of unacceptable pectoral muscle segmentations from mammograms with excellent discriminatory power.
- 22 May 2023
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 68 (11) , 115007
Open AccessUncertainty estimation for deep learning-based pectoral muscle segmentation via Monte Carlo dropout
Objective . Deep Learning models are often susceptible to failures after deployment. Knowing when your model is producing inadequate predictions is crucial. In this work, we investigate the utility of Monte Carlo (MC) dropout and the efficacy of the proposed uncertainty metric (UM) for flagging of unacceptable pectoral muscle segmentations in mammograms. Approach . Segmentation of pectoral muscle was performed with modified ResNet18 convolutional neural network. MC dropout layers were kept unlocked at inference time. For each mammogram, 50 pectoral muscle segmentations were generated. The mean was used to produce the final segmentation and the standard deviation was applied for the estimation of uncertainty. From each pectoral muscle uncertainty map, the overall UM was calculated. To validate the UM, a correlation between the dice similarity coefficient (DSC) and UM was used. The UM was first validated in a training set (200 mammograms) and finally tested in an independent dataset (300 mammograms). ROC-AUC analysis was performed to test the discriminatory power of the proposed UM for flagging unacceptable segmentations. Main results . The introduction of dropout layers in the model improved segmentation performance (DSC = 0.95 ± 0.07 versus DSC = 0.93 ± 0.10). Strong anti-correlation ( r = −0.76, p < 0.001) between the proposed UM and DSC was observed. A high AUC of 0.98 (97% specificity at 100% sensitivity) was obtained for the discrimination of unacceptable segmentations. Qualitative inspection by the radiologist revealed that images with high UM are difficult to segment. Significance . The use of MC dropout at inference time in combination with the proposed UM enables flagging of unacceptable pectoral muscle segmentations from mammograms with excellent discriminatory power.
- 30 September 2022
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 67 (19) , 195014
Development of a deep learning network for Alzheimer’s disease classification with evaluation of imaging modality and longitudinal data
Objective . Neuroimaging uncovers important information about disease in the brain. Yet in Alzheimer’s disease (AD), there remains a clear clinical need for reliable tools to extract diagnoses from neuroimages. Significant work has been done to develop deep learning (DL) networks using neuroimaging for AD diagnosis. However, no particular model has emerged as optimal. Due to a lack of direct comparisons and evaluations on independent data, there is no consensus on which modality is best for diagnostic models or whether longitudinal information enhances performance. The purpose of this work was (1) to develop a generalizable DL model to distinguish neuroimaging scans of AD patients from controls and (2) to evaluate the influence of imaging modality and longitudinal data on performance. Approach . We trained a 2-class convolutional neural network (CNN) with and without a cascaded recurrent neural network (RNN). We used datasets of 772 ( N AD = 364, N control = 408) 3D 18 F-FDG PET scans and 780 ( N AD = 280, N control = 500) T1-weighted volumetric-3D MR images (containing 131 and 144 patients with multiple timepoints) from the Alzheimer’s Disease Neuroimaging Initiative, plus an independent set of 104 ( N AD = 63, N NC = 41) 18 F-FDG PET scans (one per patient) for validation. Main Results . ROC analysis showed that PET-trained models outperformed MRI-trained, achieving maximum AUC with the CNN + RNN model of 0.93 ± 0.08, with accuracy 82.5 ± 8.9%. Adding longitudinal information offered significant improvement to performance on 18 F-FDG PET, but not on T1-MRI. CNN model validation with an independent 18 F-FDG PET dataset achieved AUC of 0.99. Layer-wise relevance propagation heatmaps added CNN interpretability. Significance . The development of a high-performing tool for AD diagnosis, with the direct evaluation of key influences, reveals the advantage of using 18 F-FDG PET and longitudinal data over MRI and single timepoint analysis. This has significant implications for the potential of neuroimaging for future research on AD diagnosis and clinical management of suspected AD patients.
- 30 September 2022
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 67 (19) , 195014
Development of a deep learning network for Alzheimer’s disease classification with evaluation of imaging modality and longitudinal data
Objective . Neuroimaging uncovers important information about disease in the brain. Yet in Alzheimer’s disease (AD), there remains a clear clinical need for reliable tools to extract diagnoses from neuroimages. Significant work has been done to develop deep learning (DL) networks using neuroimaging for AD diagnosis. However, no particular model has emerged as optimal. Due to a lack of direct comparisons and evaluations on independent data, there is no consensus on which modality is best for diagnostic models or whether longitudinal information enhances performance. The purpose of this work was (1) to develop a generalizable DL model to distinguish neuroimaging scans of AD patients from controls and (2) to evaluate the influence of imaging modality and longitudinal data on performance. Approach . We trained a 2-class convolutional neural network (CNN) with and without a cascaded recurrent neural network (RNN). We used datasets of 772 ( N AD = 364, N control = 408) 3D 18 F-FDG PET scans and 780 ( N AD = 280, N control = 500) T1-weighted volumetric-3D MR images (containing 131 and 144 patients with multiple timepoints) from the Alzheimer’s Disease Neuroimaging Initiative, plus an independent set of 104 ( N AD = 63, N NC = 41) 18 F-FDG PET scans (one per patient) for validation. Main Results . ROC analysis showed that PET-trained models outperformed MRI-trained, achieving maximum AUC with the CNN + RNN model of 0.93 ± 0.08, with accuracy 82.5 ± 8.9%. Adding longitudinal information offered significant improvement to performance on 18 F-FDG PET, but not on T1-MRI. CNN model validation with an independent 18 F-FDG PET dataset achieved AUC of 0.99. Layer-wise relevance propagation heatmaps added CNN interpretability. Significance . The development of a high-performing tool for AD diagnosis, with the direct evaluation of key influences, reveals the advantage of using 18 F-FDG PET and longitudinal data over MRI and single timepoint analysis. This has significant implications for the potential of neuroimaging for future research on AD diagnosis and clinical management of suspected AD patients.