Uncovering and mitigating bias in large, automated MRI analyses of brain development
Authors:
Preprint
Abstract
Large, population-based MRI studies of adolescents promise transformational insights into neurodevelopment and mental illness risk 1, 2 . However, MRI studies of youth are especially susceptible to motion and other artifacts 3, 4 . These artifacts may go undetected by automated quality control (QC) methods that are preferred in high-throughput imaging studies, 5 and can potentially introduce non-random noise into clinical association analyses. Here we demonstrate bias in structural MRI analyses of children due to inclusion of lower quality images, as identified through rigorous visual quality control of 11,263 T1 MRI scans obtained at age 9-10 through the Adolescent Brain Cognitive Development (ABCD) Study 6 . Compared to the best-rated images (44.9% of the sample), lower-quality images generally associated with decreased cortical thickness and increased cortical surface area measures (Cohen’s d 0.14-2.84). Variable image quality led to counterintuitive patterns in analyses that associated structural MRI and clinical measures, as inclusion of lower-quality scans altered apparent effect sizes in ways that increased risk for both false positives and negatives. Quality-related biases were partially mitigated by controlling for surface hole number, an automated index of topological complexity that differentiated lower-quality scans with good specificity at Baseline (0.81-0.93) and in 1,000 Year 2 scans (0.88-1.00). However, even among the highest-rated images, subtle topological errors occurred during image preprocessing, and their correction through manual edits significantly and reproducibly changed thickness measurements across much of the cortex (d 0.15-0.92). These findings demonstrate that inadequate QC of youth structural MRI scans can undermine advantages of large sample size to detect meaningful associations.Keywords
This publication has 41 references indexed in Scilit:
- Prospective motion correction with volumetric navigators (vNavs) reduces the bias and variance in brain morphometry induced by subject motionNeuroImage, 2016
- Trajectories of cortical thickness maturation in normal brain development — The importance of quality control proceduresNeuroImage, 2016
- A comparison of FreeSurfer-generated data with and without manual interventionFrontiers in Neuroscience, 2015
- Estimating the reproducibility of psychological scienceScience, 2015
- Head motion during MRI acquisition reduces gray matter volume and thickness estimatesNeuroImage, 2015
- Big Data and Large Sample Size: A Cautionary Note on the Potential for BiasClinical and Translational Science, 2014
- Methods and considerations for longitudinal structural brain imaging analysis across developmentDevelopmental Cognitive Neuroscience, 2014
- PROMO: Real‐time prospective motion correction in MRI using image‐based trackingMagnetic Resonance in Medicine, 2009
- Motion Artifact in Magnetic Resonance Imaging: Implications for Automated AnalysisNeuroImage, 2002
- Cortical Surface-Based AnalysisNeuroImage, 1999