Generalizability of Deep Learning Segmentation Algorithms for Automated Assessment of Cartilage Morphology and MRI Relaxometry
- 19 July 2022
- journal article
- research article
- Published by Wiley in Journal of Magnetic Resonance Imaging
- Vol. 57 (4) , 1029-1039
Abstract
Background: Deep learning (DL)‐based automatic segmentation models can expedite manual segmentation yet require resource‐intensive fine‐tuning before deployment on new datasets. The generalizability of DL methods to new datasets without fine‐tuning is not well characterized.Purpose: Evaluate the generalizability of DL‐based models by deploying pretrained models on independent datasets varying by MR scanner, acquisition parameters, and subject population.Study Type: Retrospective based on prospectively acquired data.Population: Overall test dataset: 59 subjects (26 females); Study 1: 5 healthy subjects (zero females), Study 2: 8 healthy subjects (eight females), Study 3: 10 subjects with osteoarthritis (eight females), Study 4: 36 subjects with various knee pathology (10 females).Field Strength/Sequence: A 3‐T, quantitative double‐echo steady state (qDESS).Assessment: Four annotators manually segmented knee cartilage. Each reader segmented one of four qDESS datasets in the test dataset. Two DL models, one trained on qDESS data and another on Osteoarthritis Initiative (OAI)‐DESS data, were assessed. Manual and automatic segmentations were compared by quantifying variations in segmentation accuracy, volume, and T2 relaxation times for superficial and deep cartilage.Statistical Tests: Dice similarity coefficient (DSC) for segmentation accuracy. Lin's concordance correlation coefficient (CCC), Wilcoxon rank‐sum tests, root‐mean‐squared error‐coefficient‐of‐variation to quantify manual vs. automatic T2 and volume variations. Bland–Altman plots for manual vs. automatic T2 agreement. A P value < 0.05 was considered statistically significant.Results: DSCs for the qDESS‐trained model, 0.79–0.93, were higher than those for the OAI‐DESS‐trained model, 0.59–0.79. T2 and volume CCCs for the qDESS‐trained model, 0.75–0.98 and 0.47–0.95, were higher than respective CCCs for the OAI‐DESS‐trained model, 0.35–0.90 and 0.13–0.84. Bland–Altman 95% limits of agreement for superficial and deep cartilage T2 were lower for the qDESS‐trained model, ±2.4 msec and ±4.0 msec, than the OAI‐DESS‐trained model, ±4.4 msec and ±5.2 msec.Data Conclusion: The qDESS‐trained model may generalize well to independent qDESS datasets regardless of MR scanner, acquisition parameters, and subject population.Evidence Level: 1Technical Efficacy: Stage 1Keywords
Funding Information
- National Science Foundation (GRFP‐DGE 1656518)
- GE Healthcare
- National Institutes of Health (P41 EB015891, K24 AR062068, R01‐AR074492, R01 EB002524, R00 EB022634, NIH R01‐AR077604)
This publication has 36 references indexed in Scilit:
- Quantitative measurement of femoral condyle cartilage in the knee by MRI: Validation study by multireadersJournal of Magnetic Resonance Imaging, 2013
- Quantitative MRI of articular cartilage and its clinical applicationsJournal of Magnetic Resonance Imaging, 2013
- Osteoarthritis: A disease of the joint as an organArthritis & Rheumatism, 2012
- The diagnostic performance of MRI in osteoarthritis: a systematic review and meta-analysisOsteoarthritis and Cartilage, 2012
- Evolution of semi-quantitative whole joint assessment of knee OA: MOAKS (MRI Osteoarthritis Knee Score)Osteoarthritis and Cartilage, 2011
- Intra- and inter-observer reproducibility of volume measurement of knee cartilage segmented from the OAI MR image set using a novel semi-automated segmentation methodOsteoarthritis and Cartilage, 2009
- Knee Joint: Comprehensive Assessment with 3D Isotropic Resolution Fast Spin-Echo MR Imaging—Diagnostic Performance Compared with That of Conventional MR Imaging at 3.0 TRadiology, 2009
- Rapid estimation of cartilage T2 based on double echo at steady state (DESS) with 3 TeslaMagnetic Resonance in Medicine, 2009
- Human Articular Cartilage: Influence of Aging and Early Symptomatic Degeneration on the Spatial Variation of T2—Preliminary Findings at 3 TRadiology, 2000
- A new steady‐state imaging sequence for simultaneous acquisition of two MR images with clearly different contrastsMagnetic Resonance in Medicine, 1988