The ISMRM Open Science Initiative for Perfusion Imaging (OSIPI): Results from the OSIPI–Dynamic Contrast‐Enhanced challenge
Open Access
- 19 December 2023
- journal article
- research article
- Published by Wiley in Magnetic Resonance in Medicine
- Vol. 91 (5) , 1803-1821
Abstract
Purpose: has often been proposed as a quantitative imaging biomarker for diagnosis, prognosis, and treatment response assessment for various tumors. None of the many software tools for quantification are standardized. The ISMRM Open Science Initiative for Perfusion Imaging–Dynamic Contrast‐Enhanced (OSIPI‐DCE) challenge was designed to benchmark methods to better help the efforts to standardize measurement.Methods: A framework was created to evaluate values produced by DCE‐MRI analysis pipelines to enable benchmarking. The perfusion MRI community was invited to apply their pipelines for quantification in glioblastoma from clinical and synthetic patients. Submissions were required to include the entrants' values, the applied software, and a standard operating procedure. These were evaluated using the proposed score defined with accuracy, repeatability, and reproducibility components.Results: Across the 10 received submissions, the score ranged from 28% to 78% with a 59% median. The accuracy, repeatability, and reproducibility scores ranged from 0.54 to 0.92, 0.64 to 0.86, and 0.65 to 1.00, respectively (0–1 = lowest–highest). Manual arterial input function selection markedly affected the reproducibility and showed greater variability in analysis than automated methods. Furthermore, provision of a detailed standard operating procedure was critical for higher reproducibility.Conclusions: This study reports results from the OSIPI‐DCE challenge and highlights the high inter‐software variability within estimation, providing a framework for ongoing benchmarking against the scores presented. Through this challenge, the participating teams were ranked based on the performance of their software tools in the particular setting of this challenge. In a real‐world clinical setting, many of these tools may perform differently with different benchmarking methodology.Keywords
Funding Information
- Bracco Diagnostics
This publication has 33 references indexed in Scilit:
- The secret lives of experiments: Methods reporting in the fMRI literatureNeuroImage, 2012
- Blood–Brain Barrier Permeability Abnormalities in Vascular Cognitive ImpairmentStroke, 2011
- Measurement of brain perfusion, blood volume, and blood‐brain barrier permeability, using dynamic contrast‐enhanced T1‐weighted MRI at 3 teslaMagnetic Resonance in Medicine, 2009
- Improved bolus arrival time and arterial input function estimation for tracer kinetic analysis in DCE‐MRIJournal of Magnetic Resonance Imaging, 2008
- Field strength dependence of R1 and R relaxivities of human whole blood to prohance, vasovist, and deoxyhemoglobinMagnetic Resonance in Medicine, 2008
- Age-Dependent Normal Values of T2* and T2′ in Brain ParenchymaAmerican Journal of Neuroradiology, 2008
- DCE-MRI biomarkers in the clinical evaluation of antiangiogenic and vascular disrupting agentsBritish Journal of Cancer, 2007
- Experimentally‐derived functional form for a population‐averaged high‐temporal‐resolution arterial input function for dynamic contrast‐enhanced MRIMagnetic Resonance in Medicine, 2006
- Relaxivity of Gadopentetate Dimeglumine (Magnevist), Gadobutrol (Gadovist), and Gadobenate Dimeglumine (MultiHance) in Human Blood Plasma at 0.2, 1.5, and 3 TeslaInvestigative Radiology, 2006
- Microcirculation and microvasculature in breast tumors: Pharmacokinetic analysis of dynamic MR image seriesMagnetic Resonance in Medicine, 2004