Daniel Paul Barboriak

Artificial intelligence (AI) models have shown promise in neuroradiology, yet their real-world generalizability remains uncertain, partly due to variability in imaging acquisition and protocols. We aimed to evaluate the impact of data source, scanner manufacturer, scan mode, slice thickness, and the AI models developed by participating teams on AI performance in this secondary analysis of the 2019 American Society of Functional Neuroradiology (ASFNR) AI Competition. We included 1,177 anonymized noncontrast head CT scans from five institutions. Four teams participated, developing models to detect acute ischemic stroke, intracranial hemorrhage, mass effect, and to assess age-appropriate normality. Generalized estimating equations (GEE) were used to evaluate the effects of the aforementioned variables on model performance, and collinearity diagnostics were applied to exclude redundant variables. Due to collinearity with scanner manufacturer, data source was excluded from the model. Across all tasks, the AI model employed significantly influenced performance. Scanner manufacturer was significantly associated with accuracy in detecting intracranial hemorrhage and acute ischemic stroke but not mass effect or age-based normality. Slice thickness significantly associated with detection of intracranial hemorrhage and mass effect, with thinner slices yielding higher accuracy, but showed no effect on ischemic stroke or normality assessments. Scan mode did not significantly influence performance for any task. This secondary analysis demonstrates that imaging acquisition and protocol variability may significantly affect AI model performance. Scanner manufacturer, slice thickness, and the developed AI model were significantly associated with model accuracy, whereas scan mode had no significant impact. Among these, the developed AI model consistently proved most influential, reflecting the importance of training data, model architecture, and preprocessing methods.

Radiographic assessment plays a crucial role in the management of patients with central nervous system (CNS) tumors, aiding in treatment planning and evaluation of therapeutic efficacy by quantifying response. Recently, an updated version of the Response Assessment in Neuro-Oncology (RANO) criteria (RANO 2.0) was developed to improve upon prior criteria and provide an updated, standardized framework for assessing treatment response in clinical trials for gliomas in adults. This article provides an overview of significant updates to the criteria including (1) the use of a unified set of criteria for high and low grade gliomas in adults; (2) the use of the post-radiotherapy MRI scan as the baseline for evaluation in newly diagnosed high-grade gliomas; (3) the option for the trial to mandate a confirmation scan to more reliably distinguish pseudoprogression from tumor progression; (4) the option of using volumetric tumor measurements; and (5) the removal of subjective non-enhancing tumor evaluations in predominantly enhancing gliomas (except for specific therapeutic modalities). Step-by-step pragmatic guidance is hereby provided for the neuroradiologist and imaging core lab involved in operationalization and technical execution of RANO 2.0 in clinical trials, including the display of representative cases and in-depth discussion of challenging scenarios. ABBREVIATIONS: BTIP = Brain Tumor Imaging Protocol; CE = Contrast-Enhancing; CNS = Central Nervous System; CR = Complete Response; ECOG = Eastern Cooperative Oncology Group; HGG = High-Grade Glioma; IDH = Isocitrate Dehydrogenase; IRF = Independent Radiologic Facility; LGG = Low-Grade Glioma; KPS = Karnofsky Performance Status; MR = Minor Response; mRANO = Modified RANO; NANO = Neurological Assessment in Neuro-Oncology; ORR = Objective Response Rate; OS = Overall Survival; PD = Progressive Disease; PFS = Progression-Free Survival; PR = Partial Response; PsP = Pseudoprogression; RANO = Response Assessment in Neuro-Oncology; RECIST = Response Evaluation Criteria In Solid Tumors; RT = Radiation Therapy; SD = Stable Disease; Tx = Treatment.

BACKGROUND AND PURPOSE: Artificial intelligence (AI) models in radiology are frequently developed and validated using datasets from a single institution and are rarely tested on independent, external datasets, raising questions about their generalizability and applicability in clinical practice. The American Society of Functional Neuroradiology (ASFNR) organized a multi-center AI competition to evaluate the proficiency of developed models in identifying various pathologies on NCCT, assessing age-based normality and estimating medical urgency. MATERIALS AND METHODS: In total, 1201 anonymized, full-head NCCT clinical scans from five institutions were pooled to form the dataset. The dataset encompassed normal studies as well as pathologies including acute ischemic stroke, intracranial hemorrhage, traumatic brain injury, and mass effect (detection of these-task 1). NCCTs were also assessed to determine if findings were consistent with expected brain changes for the patient9s age (task 2: age-based normality assessment) and to identify any abnormalities requiring immediate medical attention (task 3: evaluation of findings for urgent intervention). Five neuroradiologists labeled each NCCT, with consensus interpretations serving as the ground truth. The competition was announced online, inviting academic institutions and companies. Independent central analysis assessed each model9s performance. Accuracy, sensitivity, specificity, positive and negative predictive values, and receiver operating characteristic (ROC) curves were generated for each AI model, along with the area under the ROC curve (AUROC). RESULTS: 1177 studies were processed by four teams. The median age of patients was 62, with an interquartile range of 33. 19 teams from various academic institutions registered for the competition. Of these, four teams submitted their final results. No commercial entities participated in the competition. For task 1, AUROCs ranged from 0.49 to 0.59. For task 2, two teams completed the task with AUROC values of 0.57 and 0.52. For task 3, teams had little to no agreement with the ground truth. CONCLUSIONS: To assess the performance of AI models in real-world clinical scenarios, we analyzed their performance in the ASFNR AI Competition. The first ASFNR Competition underscored the gap between expectation and reality; the models largely fell short in their assessments. As the integration of AI tools into clinical workflows increases, neuroradiologists must carefully recognize the capabilities, constraints, and consistency of these technologies. Before institutions adopt these algorithms, thorough validation is essential to ensure acceptable levels of performance in clinical settings. ABBREVIATIONS: AI = artificial intelligence; ASFNR = American Society of Functional Neuroradiology; AUROC = area under the receiver operating characteristic curve; DICOM = Digital Imaging and Communications in Medicine; GEE = generalized estimation equation; IQR = interquartile range; NPV = negative predictive value; PPV = positive predictive value; ROC = receiver operating characteristic; TBI = traumatic brain injury.

PURPOSE: The Response Assessment in Neuro-Oncology (RANO) criteria for high-grade gliomas (RANO-HGG) and low-grade gliomas (RANO-LGG) were developed to improve reliability of response assessment in glioma trials. Over time, some limitations of these criteria were identified, and challenges emerged regarding integrating features of the modified RANO (mRANO) or the immunotherapy RANO (iRANO) criteria. METHODS: Informed by data from studies evaluating the different criteria, updates to the RANO criteria are proposed (RANO 2.0). RESULTS: We recommend a standard set of criteria for both high- and low-grade gliomas, to be used for all trials regardless of the treatment modalities being evaluated. In the newly diagnosed setting, the postradiotherapy magnetic resonance imaging (MRI), rather than the postsurgical MRI, will be used as the baseline for comparison with subsequent scans. Since the incidence of pseudoprogression is high in the 12 weeks after radiotherapy, continuation of treatment and confirmation of progression during this period with a repeat MRI, or histopathologic evidence of unequivocal recurrent tumor, are required to define tumor progression. However, confirmation scans are not mandatory after this period nor for the evaluation of treatment for recurrent tumors. For treatments with a high likelihood of pseudoprogression, mandatory confirmation of progression with a repeat MRI is highly recommended. The primary measurement remains the maximum cross-sectional area of tumor (two-dimensional) but volumetric measurements are an option. For IDH wild-type glioblastoma, the nonenhancing disease will no longer be evaluated except when assessing response to antiangiogenic agents. In IDH-mutated tumors with a significant nonenhancing component, clinical trials may require evaluating both the enhancing and nonenhancing tumor components for response assessment. CONCLUSION: The revised RANO 2.0 criteria refine response assessment in gliomas.

SUMMARY: In this review, concepts of algorithmic bias and fairness are defined qualitatively and mathematically. Illustrative examples are given of what can go wrong when unintended bias or unfairness in algorithmic development occurs. The importance of explainability, accountability, and transparency with respect to artificial intelligence algorithm development and clinical deployment is discussed. These are grounded in the concept of “primum no nocere” (first, do no harm). Steps to mitigate unfairness and bias in task definition, data collection, model definition, training, testing, deployment, and feedback are provided. Discussions on the implementation of fairness criteria that maximize benefit and minimize unfairness and harm to neuroradiology patients will be provided, including suggestions for neuroradiologists to consider as artificial intelligence algorithms gain acceptance into neuroradiology practice and become incorporated into routine clinical workflow.

Pediatric prolactinomas (PP) are rare but represent 50% of all pediatric pituitary adenomas [1-3]. Girls are affected more frequently than boys, although PP ten

BACKGROUND AND PURPOSE: Despite high interest in machine-learning algorithms for automated segmentation of MRIs of patients with brain tumors, there are few reports on the variability of segmentation results. The purpose of this study was to obtain benchmark measures of repeatability for a widely accessible software program, BraTumIA (Versions 1.2 and 2.0), which uses a machine-learning algorithm to segment tumor features on contrast-enhanced brain MR imaging. MATERIALS AND METHODS: Automatic segmentation of enhancing tumor, tumor edema, nonenhancing tumor, and necrosis was performed on repeat MR imaging scans obtained approximately 2 days apart in 20 patients with recurrent glioblastoma. Measures of repeatability and spatial overlap, including repeatability and Dice coefficients, are reported. RESULTS: Larger volumes of enhancing tumor were obtained on later compared with earlier scans (mean, 26.3 versus 24.2 mL for BraTumIA 1.2; P < .05; and 24.9 versus 22.9 mL for BraTumIA 2.0, P < .01). In terms of percentage change, repeatability coefficients ranged from 31% to 46% for enhancing tumor and edema components and from 87% to 116% for nonenhancing tumor and necrosis. Dice coefficients were highest (>0.7) for enhancing tumor and edema components, intermediate for necrosis, and lowest for nonenhancing tumor and did not differ between software versions. Enhancing tumor and tumor edema were smaller, and necrotic tumor larger using BraTumIA 2.0 rather than 1.2. CONCLUSIONS: Repeatability and overlap metrics varied by segmentation type, with better performance for segmentations of enhancing tumor and tumor edema compared with other components. Incomplete washout of gadolinium contrast agents could account for increasing enhancing tumor volumes on later scans.

In Radiation Therapy Oncology Group (RTOG) 0825, a phase III trial of standard therapy with bevacizumab or without (placebo) in newly diagnosed glioblastoma, 44 patients underwent dynamic contrast enhanced (DCE) and/or dynamic susceptibility contrast (DSC) MRI in the American College of Radiology Imaging Network (ACRIN) trial 6686. The association between early changes in relative cerebral blood volume (rCBV) and volume transfer constant (K^trans) with overall survival (OS) was evaluated. MRI was performed at postop baseline (S0), immediately before (S1), 1 day after (S2), and 7 weeks after (S3) bevacizumab or placebo initiation. Mean normalized and standardized rCBV (nRCBV, sRCBV) and K^trans were measured within contrast-enhancing lesion. Wilcoxon rank sum tests compared parameter changes from S1–S2 and S1–S3. Association with OS and progression-free survival (PFS) were determined using Kaplan–Meier and log-rank tests. Treatment response for groups stratified by pretreatment nRCBV (S0, S1) was explored. The intraclass correlation coefficient and repeatability coefficient for the placebo arm (S1–S2) were used to assess repeatability. Evaluable were 27–36 datasets per time point. Significant differences between treatment arms were found for changes in nRCBV and sRCBV from S1–S2 and S1–S3, and in K^trans for S1–S3. Improved PFS (P = 0.05) but not OS (P = 0.46) was observed. High pretreatment rCBV predicted improved OS for bevacizumab-treated patients. Based on the intraclass correlation coefficient, sRCBV (0.92) was more repeatable than nRCBV (0.71) and K^trans (0.75), consistent with repeatability coefficient values. Bevacizumab significantly changes rCBV but not K^trans as early as 1 day posttreatment in newly diagnosed glioblastoma unrelated to outcomes. Improvements in clinical trial design to maximize rCBV benefit are indicated.

Despite the widespread clinical use of dynamic susceptibility contrast (DSC) MRI, DSC-MRI methodology has not been standardized, hindering its utilization for response assessment in multicenter trials. Recently, the DSC-MRI Standardization Subcommittee of the Jumpstarting Brain Tumor Drug Development Coalition issued an updated consensus DSC-MRI protocol compatible with the standardized brain tumor imaging protocol (BTIP) for high-grade gliomas that is increasingly used in the clinical setting and is the default MRI protocol for the National Clinical Trials Network. After reviewing the basis for controversy over DSC-MRI protocols, this paper provides evidence-based best practices for clinical DSC-MRI as determined by the Committee, including pulse sequence (gradient echo vs spin echo), BTIP-compliant contrast agent dosing (preload and bolus), flip angle (FA), echo time (TE), and post-processing leakage correction. In summary, full-dose preload, full-dose bolus dosing using intermediate (60°) FA and field strength-dependent TE (40–50 ms at 1.5 T, 20–35 ms at 3 T) provides overall best accuracy and precision for cerebral blood volume estimates. When single-dose contrast agent usage is desired, no-preload, full-dose bolus dosing using low FA (30°) and field strength-dependent TE provides excellent performance, with reduced contrast agent usage and elimination of potential systematic errors introduced by variations in preload dose and incubation time.