Richards C. Reynolds
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Richards C. Reynolds has a total of 287 co-authors. Here are the top co-authors by number of shared publications:
- Robert W. Cox
NIMH
9 shared publications - Daniel R. Glen
Scientific and Statistical Computing Core, National Institute of Mental Health, Bethesda, USA
7 shared publications - Paul A. Taylor
National Institute of Mental Health, Bethesda, MD, USA
7 shared publications - Gang Chen
Scientific and Statistical Computing Core, National Institute of Mental Health, Bethesda, MD, USA
5 shared publications Daniel R. Glen
Scientific and Statistical Computing Core, National Institute of Mental Health, NIH, Bethesda, MD, United States
3 shared publicationsPeter A. Bandettini
Section on Functional Imaging Methods, NIMH, NIH, Bethesda, MD, USA; Functional MRI Core Facility, NIMH, NIH, Bethesda, MD, USA
3 shared publications
Latest Publications
- Open Access
- 26 August 2026
- Published by Organization for Human Brain Mapping in Aperture Neuro
- Vol. 6
Feasibility of integrating real-time fMRI neurofeedback with rTMS: Study design, preliminary results, and technical challenges
Repetitive transcranial magnetic stimulation (rTMS) is a noninvasive brain stimulation technique used in the treatment of several psychiatric disorders. However, its clinical efficacy remains limited in some cases, in part due to the lack of control over ongoing brain state during stimulation. Although some studies have combined rTMS with psychotherapy to engage the brain state during stimulation, brain state is still not objectively measured in such studies. Real-time functional magnetic resonance imaging neurofeedback (rt-fMRI-nf) offers a promising approach to objectively monitor and modulate brain state during rTMS. Here, we report the first study to investigate the feasibility of combining connectivity-based rTMS with rt-fMRI-nf to modulate amygdala activity. First, participants completed an fMRI emotion-matching task to identify an individualized medial prefrontal cortex (mPFC) target showing the strongest negative functional connectivity with the right amygdala. Across four subsequent visits, participants received active rTMS to either a mPFC target or to the vertex (control site) while engaged in rt-fMRI-nf. During rt-fMRI-nf, participants were presented with aversive images and were instructed either to passively look at them or to downregulate the height of a visual gauge representing their amygdala activity. We expected that rTMS during this downregulation would decrease amygdala activation even more than during downregulation alone. Feasibility, signal quality, rTMS-induced artifacts, and participant experience were systematically evaluated. We demonstrate that while concurrent connectivity-based rTMS and rt-fMRI-nf is feasible, the integration presents substantial technical and methodological challenges. These include limitations related to RF coil selection, signal-to-noise ratio in deep brain structures, prolonged rTMS-induced artifacts, and inter-individual variability in amygdala engagement. Despite these challenges, this work establishes a methodological foundation for future studies integrating real-time neuroimaging and neuromodulation. Continued advances in hardware, acquisition strategies, and individualized targeting may enable more precise state-dependent stimulation of emotion-related circuits, with potential implications for precision interventions in psychiatric disorders.
- 26 August 2026
- Published by Organization for Human Brain Mapping in Aperture Neuro
- Vol. 6
Open AccessFeasibility of integrating real-time fMRI neurofeedback with rTMS: Study design, preliminary results, and technical challenges
Repetitive transcranial magnetic stimulation (rTMS) is a noninvasive brain stimulation technique used in the treatment of several psychiatric disorders. However, its clinical efficacy remains limited in some cases, in part due to the lack of control over ongoing brain state during stimulation. Although some studies have combined rTMS with psychotherapy to engage the brain state during stimulation, brain state is still not objectively measured in such studies. Real-time functional magnetic resonance imaging neurofeedback (rt-fMRI-nf) offers a promising approach to objectively monitor and modulate brain state during rTMS. Here, we report the first study to investigate the feasibility of combining connectivity-based rTMS with rt-fMRI-nf to modulate amygdala activity. First, participants completed an fMRI emotion-matching task to identify an individualized medial prefrontal cortex (mPFC) target showing the strongest negative functional connectivity with the right amygdala. Across four subsequent visits, participants received active rTMS to either a mPFC target or to the vertex (control site) while engaged in rt-fMRI-nf. During rt-fMRI-nf, participants were presented with aversive images and were instructed either to passively look at them or to downregulate the height of a visual gauge representing their amygdala activity. We expected that rTMS during this downregulation would decrease amygdala activation even more than during downregulation alone. Feasibility, signal quality, rTMS-induced artifacts, and participant experience were systematically evaluated. We demonstrate that while concurrent connectivity-based rTMS and rt-fMRI-nf is feasible, the integration presents substantial technical and methodological challenges. These include limitations related to RF coil selection, signal-to-noise ratio in deep brain structures, prolonged rTMS-induced artifacts, and inter-individual variability in amygdala engagement. Despite these challenges, this work establishes a methodological foundation for future studies integrating real-time neuroimaging and neuromodulation. Continued advances in hardware, acquisition strategies, and individualized targeting may enable more precise state-dependent stimulation of emotion-related circuits, with potential implications for precision interventions in psychiatric disorders.
- 12 November 2024
- Published by MIT Press in Imaging Neuroscience
- Vol. 2, 1-52
Processing, evaluating, and understanding FMRI data with afni_proc.py
FMRI data are noisy, complicated to acquire, and typically go through many steps of processing before they are used in a study or clinical practice. Being able to visualize and understand the data from the start through the completion of processing, while being confident that each intermediate step was successful, is challenging. AFNI’s afni_proc.py is a tool to create and run a processing pipeline for FMRI data. With its flexible features, afni_proc.py allows users to both control and evaluate their processing at a detailed level. It has been designed to keep users informed about all processing steps: it does not just process the data, but also first outputs a fully commented processing script that the users can read, query, interpret, and refer back to. Having this full provenance is important for being able to understand each step of processing; it also promotes transparency and reproducibility by keeping the record of individual-level processing and modeling specifics in a single, shareable place. Additionally, afni_proc.py creates pipelines that contain several automatic self-checks for potential problems during runtime. The output directory contains a dictionary of relevant quantities that can be programmatically queried for potential issues and a systematic, interactive quality control (QC) HTML. All of these features help users evaluate and understand their data and processing in detail. We describe these and other aspects of afni_proc.py here using a set of task-based and resting-state FMRI example commands.
- 12 November 2024
- Published by MIT Press in Imaging Neuroscience
- Vol. 2, 1-52
Processing, evaluating, and understanding FMRI data with afni_proc.py
FMRI data are noisy, complicated to acquire, and typically go through many steps of processing before they are used in a study or clinical practice. Being able to visualize and understand the data from the start through the completion of processing, while being confident that each intermediate step was successful, is challenging. AFNI’s afni_proc.py is a tool to create and run a processing pipeline for FMRI data. With its flexible features, afni_proc.py allows users to both control and evaluate their processing at a detailed level. It has been designed to keep users informed about all processing steps: it does not just process the data, but also first outputs a fully commented processing script that the users can read, query, interpret, and refer back to. Having this full provenance is important for being able to understand each step of processing; it also promotes transparency and reproducibility by keeping the record of individual-level processing and modeling specifics in a single, shareable place. Additionally, afni_proc.py creates pipelines that contain several automatic self-checks for potential problems during runtime. The output directory contains a dictionary of relevant quantities that can be programmatically queried for potential issues and a systematic, interactive quality control (QC) HTML. All of these features help users evaluate and understand their data and processing in detail. We describe these and other aspects of afni_proc.py here using a set of task-based and resting-state FMRI example commands.
- Top Cited PapersOpen Access
- 26 April 2023
- Published by Elsevier in NeuroImage
- Vol. 274, 120138
Highlight results, don't hide them: Enhance interpretation, reduce biases and improve reproducibility
- 26 April 2023
- Published by Elsevier in NeuroImage
- Vol. 274, 120138
Top Cited PapersOpen AccessHighlight results, don't hide them: Enhance interpretation, reduce biases and improve reproducibility
- Open Access
- 30 January 2023
- Published by Frontiers Media SA in Frontiers in Neuroscience
- Vol. 16, 1073800
Quality control practices in FMRI analysis: Philosophy, methods and examples using AFNI
Quality control (QC) is a necessary, but often an under-appreciated, part of FMRI processing. Here we describe procedures for performing QC on acquired or publicly available FMRI datasets using the widely used AFNI software package. This work is part of the Research Topic, “Demonstrating Quality Control (QC) Procedures in fMRI.” We used a sequential, hierarchical approach that contained the following major stages: (1) GTKYD (getting to know your data, esp. its basic acquisition properties), (2) APQUANT (examining quantifiable measures, with thresholds), (3) APQUAL (viewing qualitative images, graphs, and other information in systematic HTML reports) and (4) GUI (checking features interactively with a graphical user interface); and for task data, and (5) STIM (checking stimulus event timing statistics). We describe how these are complementary and reinforce each other to help researchers stay close to their data. We processed and evaluated the provided, publicly available resting state data collections (7 groups, 139 total subjects) and task-based data collection (1 group, 30 subjects). As specified within the Topic guidelines, each subject’s dataset was placed into one of three categories: Include, exclude or uncertain. The main focus of this paper, however, is the detailed description of QC procedures: How to understand the contents of an FMRI dataset, to check its contents for appropriateness, to verify processing steps, and to examine potential quality issues. Scripts for the processing and analysis are freely available.
- 30 January 2023
- Published by Frontiers Media SA in Frontiers in Neuroscience
- Vol. 16, 1073800
Open AccessQuality control practices in FMRI analysis: Philosophy, methods and examples using AFNI
Quality control (QC) is a necessary, but often an under-appreciated, part of FMRI processing. Here we describe procedures for performing QC on acquired or publicly available FMRI datasets using the widely used AFNI software package. This work is part of the Research Topic, “Demonstrating Quality Control (QC) Procedures in fMRI.” We used a sequential, hierarchical approach that contained the following major stages: (1) GTKYD (getting to know your data, esp. its basic acquisition properties), (2) APQUANT (examining quantifiable measures, with thresholds), (3) APQUAL (viewing qualitative images, graphs, and other information in systematic HTML reports) and (4) GUI (checking features interactively with a graphical user interface); and for task data, and (5) STIM (checking stimulus event timing statistics). We describe how these are complementary and reinforce each other to help researchers stay close to their data. We processed and evaluated the provided, publicly available resting state data collections (7 groups, 139 total subjects) and task-based data collection (1 group, 30 subjects). As specified within the Topic guidelines, each subject’s dataset was placed into one of three categories: Include, exclude or uncertain. The main focus of this paper, however, is the detailed description of QC procedures: How to understand the contents of an FMRI dataset, to check its contents for appropriateness, to verify processing steps, and to examine potential quality issues. Scripts for the processing and analysis are freely available.
- Open Access
Highlight Results, Don’t Hide Them: Enhance interpretation, reduce biases and improve reproducibility
Most neuroimaging studies display results that represent only a tiny fraction of the collected data. While it is conventional to present “only the significant results” to the reader, here we suggest that this practice has several negative consequences for both reproducibility and understanding. This practice hides away most of the results of the dataset and leads to problems of selection bias and irreproducibility, both of which have been recognized as major issues in neuroimaging studies recently. Opaque, all-or-nothing thresholding, even if well-intentioned, places undue influence on arbitrary filter values, hinders clear communication of scientific results, wastes data, is antithetical to good scientific practice, and leads to conceptual inconsistencies. It is also inconsistent with the properties of the acquired data and the underlying biology being studied. Instead of presenting only a few statistically significant locations and hiding away the remaining results, we propose that studies should “highlight” the former while also showing as much as possible of the rest. This is distinct from but complementary to utilizing data sharing repositories: the initial presentation of results has an enormous impact on the interpretation of a study. We present practical examples for voxelwise, regionwise and cross-study analyses using publicly available data that was analyzed previously by 70 teams (NARPS; Botvinik-Nezer, et al., 2020), showing that it is possible to balance the goals of displaying a full set of results with providing the reader reasonably concise and “digestible” findings. In particular, the highlighting approach sheds useful light on the kind of variability present among the NARPS teams’ results, which is primarily a varied strength of agreement rather than disagreement. Using a meta-analysis built on the informative “highlighting” approach shows this relative agreement, while one using the standard “hiding” approach does not. We describe how this simple but powerful change in practice---focusing on highlighting results, rather than hiding all but the strongest ones---can help address many large concerns within the field, or at least to provide more complete information about them. We include a list of practical suggestions for results reporting to improve reproducibility, cross-study comparisons and meta-analyses. Highlights: Most studies do not present all results of their analysis, hiding subthreshold ones. Hiding results negatively affects the interpretation and understanding of the study. Neuroimagers should present all results of their study, highlighting key ones. Using the public NARPS data, we show several benefits of the “highlighting” approach. The highlighting approach improves individual studies and meta-analyses.
Open AccessHighlight Results, Don’t Hide Them: Enhance interpretation, reduce biases and improve reproducibility
Most neuroimaging studies display results that represent only a tiny fraction of the collected data. While it is conventional to present “only the significant results” to the reader, here we suggest that this practice has several negative consequences for both reproducibility and understanding. This practice hides away most of the results of the dataset and leads to problems of selection bias and irreproducibility, both of which have been recognized as major issues in neuroimaging studies recently. Opaque, all-or-nothing thresholding, even if well-intentioned, places undue influence on arbitrary filter values, hinders clear communication of scientific results, wastes data, is antithetical to good scientific practice, and leads to conceptual inconsistencies. It is also inconsistent with the properties of the acquired data and the underlying biology being studied. Instead of presenting only a few statistically significant locations and hiding away the remaining results, we propose that studies should “highlight” the former while also showing as much as possible of the rest. This is distinct from but complementary to utilizing data sharing repositories: the initial presentation of results has an enormous impact on the interpretation of a study. We present practical examples for voxelwise, regionwise and cross-study analyses using publicly available data that was analyzed previously by 70 teams (NARPS; Botvinik-Nezer, et al., 2020), showing that it is possible to balance the goals of displaying a full set of results with providing the reader reasonably concise and “digestible” findings. In particular, the highlighting approach sheds useful light on the kind of variability present among the NARPS teams’ results, which is primarily a varied strength of agreement rather than disagreement. Using a meta-analysis built on the informative “highlighting” approach shows this relative agreement, while one using the standard “hiding” approach does not. We describe how this simple but powerful change in practice---focusing on highlighting results, rather than hiding all but the strongest ones---can help address many large concerns within the field, or at least to provide more complete information about them. We include a list of practical suggestions for results reporting to improve reproducibility, cross-study comparisons and meta-analyses. Highlights: Most studies do not present all results of their analysis, hiding subthreshold ones. Hiding results negatively affects the interpretation and understanding of the study. Neuroimagers should present all results of their study, highlighting key ones. Using the public NARPS data, we show several benefits of the “highlighting” approach. The highlighting approach improves individual studies and meta-analyses.
- Open Access
Instructions and experiential learning have similar impacts on pain and pain-related brain responses but produce dissociations in value-based reversal learning
Recent data suggest that interactions between systems involved in higher order knowledge and associative learning drive responses during appetitive and aversive learning. However, it is unknown how these systems impact subjective responses, such as pain. We tested how instructions and reversal learning influence pain and pain-evoked brain activation. Healthy volunteers (n = 40) were either instructed about contingencies between cues and aversive outcomes or learned through experience in a paradigm where contingencies reversed three times. We measured predictive cue effects on pain and heat-evoked brain responses using functional magnetic resonance imaging. Predictive cues dynamically modulated pain perception as contingencies changed, regardless of whether participants received contingency instructions. Heat-evoked responses in the insula, anterior cingulate, and putamen updated as contingencies changed, whereas the periaqueductal gray and thalamus responded to initial contingencies throughout the task. Quantitative modeling revealed that expected value was shaped purely by instructions in the Instructed Group, whereas expected value updated dynamically in the Uninstructed Group as a function of error-based learning. These differences were accompanied by dissociations in the neural correlates of value-based learning in the rostral anterior cingulate, medial prefrontal cortex, and orbitofrontal cortex. These results show how predictions impact subjective pain. Moreover, imaging data delineate three types of networks involved in pain generation and value-based learning: those that respond to initial contingencies, those that update dynamically during feedback-driven learning as contingencies change, and those that are sensitive to instruction. Together, these findings provide multiple points of entry for therapies designs to impact pain.
Open AccessInstructions and experiential learning have similar impacts on pain and pain-related brain responses but produce dissociations in value-based reversal learning
Recent data suggest that interactions between systems involved in higher order knowledge and associative learning drive responses during appetitive and aversive learning. However, it is unknown how these systems impact subjective responses, such as pain. We tested how instructions and reversal learning influence pain and pain-evoked brain activation. Healthy volunteers (n = 40) were either instructed about contingencies between cues and aversive outcomes or learned through experience in a paradigm where contingencies reversed three times. We measured predictive cue effects on pain and heat-evoked brain responses using functional magnetic resonance imaging. Predictive cues dynamically modulated pain perception as contingencies changed, regardless of whether participants received contingency instructions. Heat-evoked responses in the insula, anterior cingulate, and putamen updated as contingencies changed, whereas the periaqueductal gray and thalamus responded to initial contingencies throughout the task. Quantitative modeling revealed that expected value was shaped purely by instructions in the Instructed Group, whereas expected value updated dynamically in the Uninstructed Group as a function of error-based learning. These differences were accompanied by dissociations in the neural correlates of value-based learning in the rostral anterior cingulate, medial prefrontal cortex, and orbitofrontal cortex. These results show how predictions impact subjective pain. Moreover, imaging data delineate three types of networks involved in pain generation and value-based learning: those that respond to initial contingencies, those that update dynamically during feedback-driven learning as contingencies change, and those that are sensitive to instruction. Together, these findings provide multiple points of entry for therapies designs to impact pain.
- Open Access
- 1 August 2021
- Published by Elsevier in NeuroImage
- Vol. 237, 118091
LayNii: A software suite for layer-fMRI
- 1 August 2021
- Published by Elsevier in NeuroImage
- Vol. 237, 118091
Open AccessLayNii: A software suite for layer-fMRI
- Open Access
EPI Distortion Correction is Easy and Useful, and You Should Use It: A case study with toddler data
Task, resting state, and diffusion MRI data are usually acquired from subjects using echo-planar based imaging techniques. These techniques are highly susceptible to B 0 homogeneity effects that result in geometric distortions in the reconstructed images. As researchers work to link the information from these scans back to various developmental stages, or to conditions and diseases in specific regions or structures of the brain, it becomes critical to have accurate correspondence between more geometrically distorted echo-planar images and less geometrically distorted anatomical images. A variety of techniques and tools have been developed to improve this correspondence. The basic premise behind most techniques used to mitigate geometric distortion is to acquire enough information to inform software tools how echo-planar images are warped, then have them undo that warping. Here, we investigate the application of two common methods: B 0 correction, and reverse-polarity phase-encoding (or reverse blip) correction. We implement each of these in two separate, widely used software packages in the field: AFNI and FSL. We find that using either technique in either software package results in reduced geometric distortions in the EPI images. We discuss the practical implementations of these methods (e.g., increased scan and processing time). In general, however, both methods possess readily available data acquisition schemes, and are highly efficient to include in processing streams. Due to the overall data improvement, we strongly recommend that researchers include one of these methods in their standard protocols.
Open AccessEPI Distortion Correction is Easy and Useful, and You Should Use It: A case study with toddler data
Task, resting state, and diffusion MRI data are usually acquired from subjects using echo-planar based imaging techniques. These techniques are highly susceptible to B 0 homogeneity effects that result in geometric distortions in the reconstructed images. As researchers work to link the information from these scans back to various developmental stages, or to conditions and diseases in specific regions or structures of the brain, it becomes critical to have accurate correspondence between more geometrically distorted echo-planar images and less geometrically distorted anatomical images. A variety of techniques and tools have been developed to improve this correspondence. The basic premise behind most techniques used to mitigate geometric distortion is to acquire enough information to inform software tools how echo-planar images are warped, then have them undo that warping. Here, we investigate the application of two common methods: B 0 correction, and reverse-polarity phase-encoding (or reverse blip) correction. We implement each of these in two separate, widely used software packages in the field: AFNI and FSL. We find that using either technique in either software package results in reduced geometric distortions in the EPI images. We discuss the practical implementations of these methods (e.g., increased scan and processing time). In general, however, both methods possess readily available data acquisition schemes, and are highly efficient to include in processing streams. Due to the overall data improvement, we strongly recommend that researchers include one of these methods in their standard protocols.
- Open Access
LayNii: A software suite for layer-fMRI
High-resolution fMRI in the sub-millimeter regime allows researchers to resolve brain activity across cortical layers and columns non-invasively. While these high-resolution data make it possible to address novel questions of directional information flow within and across brain circuits, the corresponding data analyses are challenged by MRI artifacts, including image blurring, image distortions, low SNR, and restricted coverage. These challenges often result in insufficient spatial accuracy of conventional analysis pipelines. Here we introduce a new software suite that is specifically designed for layer-specific functional MRI: LayNii. This toolbox is a collection of command-line executable programs written in C/C++ and is distributed open-source and as pre-compiled binaries for Linux, Windows, and macOS. LayNii is designed for layer-fMRI data that suffer from SNR and coverage constraints and thus cannot be straightforwardly analyzed in alternative software packages. Some of the most popular programs of LayNii contain ‘layerification’ and columnarization in the native voxel space of functional data as well as many other layer-fMRI specific analysis tasks: layer-specific smoothing, model-based vein mitigation of GE-BOLD data, quality assessment of artifact dominated sub-millimeter fMRI, as well as analyses of VASO data. Highlights: A new software toolbox is introduced for layer-specific functional MRI: LayNii. LayNii is a suite of command-line executable C++ programs for Linux, Windows, and macOS. LayNii is designed for layer-fMRI data that suffer from SNR and coverage constraints. LayNii performs layerification in the native voxel space of functional data. LayNii performs layer-smoothing, GE-BOLD deveining, QA, and VASO analysis. Abstract Figure: Graphical abstract
Open AccessLayNii: A software suite for layer-fMRI
High-resolution fMRI in the sub-millimeter regime allows researchers to resolve brain activity across cortical layers and columns non-invasively. While these high-resolution data make it possible to address novel questions of directional information flow within and across brain circuits, the corresponding data analyses are challenged by MRI artifacts, including image blurring, image distortions, low SNR, and restricted coverage. These challenges often result in insufficient spatial accuracy of conventional analysis pipelines. Here we introduce a new software suite that is specifically designed for layer-specific functional MRI: LayNii. This toolbox is a collection of command-line executable programs written in C/C++ and is distributed open-source and as pre-compiled binaries for Linux, Windows, and macOS. LayNii is designed for layer-fMRI data that suffer from SNR and coverage constraints and thus cannot be straightforwardly analyzed in alternative software packages. Some of the most popular programs of LayNii contain ‘layerification’ and columnarization in the native voxel space of functional data as well as many other layer-fMRI specific analysis tasks: layer-specific smoothing, model-based vein mitigation of GE-BOLD data, quality assessment of artifact dominated sub-millimeter fMRI, as well as analyses of VASO data. Highlights: A new software toolbox is introduced for layer-specific functional MRI: LayNii. LayNii is a suite of command-line executable C++ programs for Linux, Windows, and macOS. LayNii is designed for layer-fMRI data that suffer from SNR and coverage constraints. LayNii performs layerification in the native voxel space of functional data. LayNii performs layer-smoothing, GE-BOLD deveining, QA, and VASO analysis. Abstract Figure: Graphical abstract
- Top Cited Papers
- 20 May 2020
- Published by Springer Nature in Nature
- Vol. 582 (7810) , 84-88
Variability in the analysis of a single neuroimaging dataset by many teams
Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2,3,4,5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.
- 20 May 2020
- Published by Springer Nature in Nature
- Vol. 582 (7810) , 84-88
Top Cited PapersVariability in the analysis of a single neuroimaging dataset by many teams
Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2,3,4,5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.