Michael W. Ruff

The MRI model was run on 34 patients with tumefactive demyelination. (A) Distribution of predicted MRI score for the 34 patients. (B) Distribution of predicted MRI score by age at diagnosis. The blue line denotes a loess fit and the grey shaded area denotes the 95% confidence interval. (C) Distribution of predicted MRI score by gender (F=female, M=Male). (D) Distribution of predicted MRI score by MRI manufacturer. (E) Distribution of predicted MRI score by MRI field strength. (F) Distribution of predicted MRI score by T1Gd acquisition type. T2 acquisition type is not shown because 33 of the 34 patients were sequenced using 2D.

Model performance on the 256 GBM and 73 CNS-DLBCL independent test cases from stage 3 from the model developed using the ensemble approach (A) overall, (B) stratified by sex, and (C) stratified by age. Results for the model developed using the loss approach (D) overall, (E) stratified by sex, and (F) stratified by age. AUCs (95% CI) are provided.

Glioblastoma (GBM), isocitrate dehydrogenase wild-type (IDHwt) and central nervous system diffuse large B-cell lymphoma (CNS-DLBCL) are aggressive brain tumors with overlapping MRI features, yet distinct treatment approaches. Noninvasive tools are needed to aid in differential diagnosis. Deep learning on T1 postcontrast and T2-weighted MRI sequences were used to differentiate GBM and CNS-DLBCL. A three-stage temporal study design was utilized. Model development was performed on 146 patients with CNS-DLBCL and 146 age-matched, sex-matched, and MRI year–matched patients with GBM diagnosed at Mayo Clinic between 1998 and 2019. Models were tested on independent temporal test cohorts. Initial testing included 240 independent GBM diagnosed at Mayo Clinic between 1998 and 2019. The prospective test cohort included 37 patients with CNS-DLBCL and 256 patients with GBM diagnosed at Mayo Clinic after January 1, 2020, and 36 patients with CNS-DLBCL diagnosed at an external institution. Of the patients diagnosed at Mayo Clinic, 47% had MRIs generated from non-Mayo institutions. Two different model approaches were compared: (i) ensemble approach using area under the receiver operating characteristic curve (AUC) and cross-validation for model selection and (ii) loss approach minimizing cross-entropy loss and cross-validation to evaluate prediction performance. The AUCs on the prospective test cohort were 0.84 [95% confidence interval (CI), 0.78–0.90] and 0.83 (95% CI, 0.77–0.88) for the ensemble and loss approaches, respectively. Stability of ensemble prediction improved with the increasing number of models. Stratified AUC analysis demonstrated consistent performance across sex and age. We utilized a robust temporal study design and applied 2 different analytic approaches to develop a classification model. The findings confirm the feasibility of using MRI-based deep learning models to differentiate GBM from CNS-DLBCL.Significance:GBM, IDHwt and CNS-DLBCL are aggressive brain tumors with overlapping MRI features, yet distinct treatment approaches. Noninvasive tools are needed to aid in differential diagnosis. We developed MRI-based deep learning models to differentiate GBM, IDHwt from CNS-DLBCL using a rigorous three-stage temporal design that included prospective validation. The model AUC on a prospective cohort was 0.84.

Cartoon comparing AUC and cross entropy loss on five GBM patients and five CNS-DLBCL cases. Grey denotes GBM and black denotes CNS-DLBCL. All models have the same AUC (AUC=100%); however, Model 4 has the smallest cross entropy loss.

Violin plots showing the model scores for the 256 GBM and 73 CNS-DLBCL cases in stage 3, obtained from the models developed using the (A) ensemble approach and (B) loss approach.

Evaluating the effect on model performance (AUC) when performing ensemble across an increasingly larger number of models. Five independent sets of 5-fold cross-validation were evaluated. One 5-fold cross-validation set denotes performing ensemble across the resulting k = 5 models. Two 5-fold cross-validation sets denote ensemble across 10 models, three sets denote ensemble across 15 models, four sets denote ensemble across 20 models, and five sets denote ensemble across 25 models.

Sensitivity and specificity of the loss model applied to the stage 3 cohort. All possible model score thresholds from the loss model are shown. The yellow highlighted row denotes the score threshold required to obtain 90% sensitivity. The orange highlighted row denotes the score threshold required to obtain 90% specificity.

Description of the three-stage study design.

Demographic and MRI characteristics of patients involved in stage 1, stage 2, and stage 3.

Violin plots showing the model scores from the 240 retrospective GBM cases in stage 2 from the models developed using the (A) ensemble approach and (B) loss approach.