ULTRA-SR Challenge: Assessment of Ultrasound Localization and TRacking Algorithms for Super-Resolution Imaging
- 12 April 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 43 (8) , 2970-2987
Abstract
With the widespread interest and uptake of super-resolution ultrasound (SRUS) through localization and tracking of microbubbles, also known as ultrasound localization microscopy (ULM), many localization and tracking algorithms have been developed. ULM can image many centimeters into tissue in-vivo and track microvascular flow non-invasively with sub-diffraction resolution. In a significant community effort, we organized a challenge, Ultrasound Localization and TRacking Algorithms for Super-Resolution (ULTRA-SR). The aims of this paper are threefold: to describe the challenge organization, data generation, and winning algorithms; to present the metrics and methods for evaluating challenge entrants; and to report results and findings of the evaluation. Realistic ultrasound datasets containing microvascular flow for different clinical ultrasound frequencies were simulated, using vascular flow physics, acoustic field simulation and nonlinear bubble dynamics simulation. Based on these datasets, 38 submissions from 24 research groups were evaluated against ground truth using an evaluation framework with six metrics, three for localization and three for tracking. In-vivo mouse brain and human lymph node data were also provided, and performance assessed by an expert panel. Winning algorithms are described and discussed. The publicly available data with ground truth and the defined metrics for both localization and tracking present a valuable resource for researchers to benchmark algorithms and software, identify optimized methods/software for their data, and provide insight into the current limits of the field. In conclusion, Ultra-SR challenge has provided benchmarking data and tools as well as direct comparison and insights for a number of the state-of-the art localization and tracking algorithms.Keywords
Funding Information
- U.K. Engineering and Physical Sciences Research Council (EP/T008970/1, EP/V04799X/1)
- Chan Zuckerberg Foundation (2020-225443)
- EPSRC Impact Acceleration Account Funding (EP/R511547/1)
- Medical Research Council (MRC) Confidence in Concept Scheme at Imperial College (MC-PC-18050)
- Chief Scientist Office Scotland (TCS/18/40)
- Medical Research Council Confidence in Concept (MRC/CIC3/027)
- Science and Technology Facilities Council Project (ST/M007804/1)
- British Heart Foundation (FS/PhD/21/29113)
- European Research Council (ERC) under the European Union Horizon H2020 Program/ERC Consolidator (772786-ResolveStroke)
- Natural Sciences and Engineering Research Council of Canada
- Deutsche Forschungsgemeinschaft (233312120, 445703531)
- Canada Research Chair Program
- Burroughs Wellcome Fund (1018212.03)
- European Research Council (ERC), Synergy, through SUper Resolution ultrasound imaging using Erythrocytes (854796)
- National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health (R21EB030072)
- Beckman Institute Postdoctoral Fellowship
- FV Hunt Postdoctoral Research Fellowship in Acoustics from the Acoustical Society of America
- National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health (R21EB030072)
This publication has 54 references indexed in Scilit:
- Context-aware deep learning enables high-efficacy localization of high concentration microbubbles for super-resolution ultrasound localization microscopyNature Communications, 2024
- Coronary Flow Assessment Using 3-Dimensional Ultrafast Ultrasound Localization MicroscopyJACC: Cardiovascular Imaging, 2022
- Ultrasound super-resolution imaging with a hierarchical Kalman trackerUltrasonics, 2022
- Ultrasound super-resolution imaging provides a noninvasive assessment of renal microvasculature changes during mouse acute kidney injuryKidney International, 2020
- Super-resolution Ultrasound ImagingUltrasound in Medicine & Biology, 2020
- Motion model ultrasound localization microscopy for preclinical and clinical multiparametric tumor characterizationNature Communications, 2018
- Subwavelength motion-correction for ultrafast ultrasound localization microscopyUltrasonics, 2017
- Ultrafast ultrasound localization microscopy for deep super-resolution vascular imagingNature, 2015
- The Hungarian method for the assignment problemNaval Research Logistics (NRL), 2004
- Microbubbles in medical imaging: current applications and future directionsNature Reviews Drug Discovery, 2004