RGRaphael T. Gerraty

Raphael T. Gerraty

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Raphael T. Gerraty has a total of 57 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Two routes to value-based decisions in Parkinson’s disease: differentiating incremental reinforcement learning from episodic memory

    by Leila Montaser-Kouhsari, Jonathan Nicholas, Raphael T. Gerraty, Daphna Shohamy

    Patients with Parkinson’s disease are impaired at incremental reward-based learning. It is typically assumed that this impairment reflects a loss of striatal dopamine. However, many open questions remain about the nature of reward-based learning deficits in Parkinson’s. Recent studies have found that a combination of different cognitive and computational strategies contribute even to simple reward-based learning tasks, suggesting a possible role for episodic memory. These findings raise critical questions about how incremental learning and episodic memory interact to support learning from past experience and what their relative contributions are to impaired decision-making in Parkinson’s disease. Here we addressed these questions by asking patients with Parkinson’s disease (n=26) both on and off their dopamine replacement medication and age- and education-matched healthy controls (n=26) to complete a task designed to isolate the contributions of incremental learning and episodic memory to reward-based learning and decision-making. We found that Parkinson’s patients performed as well as healthy controls when using episodic memory, but were impaired at incremental reward-based learning. Dopamine replacement medication remediated this deficit while enhancing subsequent episodic memory for the value of motivationally relevant stimuli. These results demonstrate that Parkinson’s patients are impaired at learning about reward from trial-and-error when episodic memory is properly controlled for, and that learning based on the value of single experiences remains intact in patients with Parkinson’s disease.


  • Open Access

    Source Invariance and Probabilistic Transfer: A Testable Theory of Probabilistic Neural Representations

    by Samuel Lippl, Raphael Gerraty, John Morrison, Nikolaus Kriegeskorte

    As animals interact with their environments, they must infer properties of their surroundings. Some animals, including humans, can represent uncertainty about those properties. But when, if ever, do they use probability distributions to represent their uncertainty? It depends on which definition we choose. In this paper, we argue that existing definitions are inadequate because they are untestable. We then propose our own definition. There are two reasons why existing definitions are untestable. First, they do not distinguish between representations of uncertainty and representations of variables merely related to uncertainty ('representational indeterminacy'). Second, they do not distinguish between probabilistic representations of uncertainty and merely "heuristic" representations of uncertainty. We call this 'model indeterminacy' because the underlying problem is that we do not have access to the animal's generative model. We define probabilistic representations by two properties: 1) they encode uncertainty regardless of the source of the uncertainty ('source invariance'), 2) they support the efficient learning of new tasks that would be more difficult to learn given non-probabilistic representations ('probabilistic task transfer'). Source invariance indicates that they are representations of uncertainty rather than variables merely related to uncertainty, thereby solving representational indeterminacy. Probabilistic task transfer indicates that they are probabilistic representations of uncertainty rather than merely heuristic representations, thereby solving model indeterminacy.

  • Open Access

    Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T

    by Jaron T. Colas, Neil M. Dundon, Raphael T. Gerraty, Natalie M. Saragosa‐Harris, Karol P. Szymula, Koranis Tanwisuth, J. Michael Tyszka, Camilla van Geen, Harang Ju, Arthur W. Toga, et al.

    The model‐free algorithms of “reinforcement learning” (RL) have gained clout across disciplines, but so too have model‐based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This “generalized reinforcement learning” (GRL) model, a frugal extension of RL, parsimoniously retains the single reward‐prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal‐learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high‐resolution high‐field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value‐based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.


  • Open Access

    Probabilistic representations should be defined by their computational role: reply to Rahnev et al. 2021

    by Raphael Gerraty, Samuel Lippl, John Morrison, Nikolaus Kriegeskorte

    Note: This is a reply to the Generative Adversarial Collaboration "Is perception probabilistic? Clarifying the definitions" by Rahnev, Block, Denison, and Jehee (DOI 10.31234/osf.io/f8v5r). Probabilistic representations promise to shed light on how humans and other organisms cope with uncertainty, but debates about them have been hindered by unclear definitions and a lack of testable predictions. We argue that neither camp in this GAC has offered a plausible way out of this situation. We describe problems with their attempts to define probabilistic representations and sketch an alternative view, in which the defining feature of a probabilistic representation is how it is used.

  • Open Access

    Hierarchical Bayesian Models of Reinforcement Learning: Introduction and comparison to alternative methods

    by Camilla van Geen, Raphael T. Gerraty

    Reinforcement learning models have been used extensively to capture learning and decision-making processes in humans and other organisms. One essential goal of these computational models is the generalization to new sets of observations. Extracting parameters that can reliably predict out-of-sample data can be difficult, however. The use of prior distributions to regularize parameter estimates has been shown to help remedy this issue. While previous research has suggested that empirical priors estimated from a separate dataset improve predictive accuracy, this paper outlines an alternate method for the derivation of empirical priors: hierarchical Bayesian modeling. We provide a detailed introduction to this method, and show that using hierarchical models to simultaneously extract and impose empirical priors leads to better out-of-sample prediction while being more data efficient.


  • Open Access

    Dopamine modulates learning-related changes in dynamic striatal-cortical connectivity in Parkinson’s disease

    by Raphael T. Gerraty, Madeleine E. Sharp, Amanda Buch, Danielle S. Bassett, Daphna Shohamy

    Learning from reinforcement is thought to depend on striatal dopamine inputs, which serve to update the value of actions by modifying connections in widespread cortico-striatal circuits. While considerable research has described the activity of individual striatal and midbrain regions in reinforcement learning, the broader role for dopamine in modulating network-level processes has been difficult to decipher. To examine whether dopamine modulates circuit-level dynamic connectivity during learning, we characterized the effects of dopamine on learning-related dynamic functional connectivity estimated from fMRI data acquired in patients with Parkinson’s disease. Patients with Parkinson’s disease have severe dopamine depletion in the striatum and are treated with dopamine replacement drugs, providing an opportunity to compare learning and network dynamics when patients are in a low dopamine state (off drugs) versus a high dopamine state (on drugs). We assessed the relationship between dopamine and dynamic connectivity while patients performed a probabilistic reversal learning task. We found that reversal learning altered dynamic network flexibility in the striatum and that this effect was dependent on dopaminergic state. We also found that dopamine modulated changes in connectivity between the striatum and specific task-relevant visual areas of inferior temporal cortex, providing empirical support for theories stipulating that value is updated through changes in cortico-striatal circuits. These results suggest that dopamine exerts a widespread effect on neural circuitry and network dynamics during reinforcement learning.


  • Open Access

    Dynamic Flexibility in Striatal-Cortical Circuits Supports Reinforcement Learning

    by Raphael T. Gerraty, Juliet Y. Davidow, Karin Foerde, Adriana Galvan, Danielle S. Bassett, Daphna Shohamy

    Complex learned behaviors must involve the integrated action of distributed brain circuits. Although the contributions of individual regions to learning have been extensively investigated, much less is known about how distributed brain networks orchestrate their activity over the course of learning. To address this gap, we used fMRI combined with tools from dynamic network neuroscience to obtain time-resolved descriptions of network coordination during reinforcement learning in humans. We found that learning to associate visual cues with reward involves dynamic changes in network coupling between the striatum and distributed brain regions, including visual, orbitofrontal, and ventromedial prefrontal cortex (n= 22; 13 females). Moreover, we found that this flexibility in striatal network coupling correlates with participants' learning rate and inverse temperature, two parameters derived from reinforcement learning models. Finally, we found that episodic learning, measured separately in the same participants at the same time, was related to dynamic connectivity in distinct brain networks. These results suggest that dynamic changes in striatal-centered networks provide a mechanism for information integration during reinforcement learning.SIGNIFICANCE STATEMENTLearning from the outcomes of actions, referred to asreinforcement learning, is an essential part of life. The roles of individual brain regions in reinforcement learning have been well characterized in terms of updating values for actions or cues. Missing from this account, however, is an understanding of how different brain areas interact during learning to integrate sensory and value information. Here we characterize flexible striatal-cortical network dynamics that relate to reinforcement learning behavior.


  • Open Access

    Dynamic flexibility in striatal-cortical circuits supports reinforcement learning

    by Raphael T. Gerraty, Juliet Y. Davidow, Karin Foerde, Adriana Galvan, Danielle S. Bassett, Daphna Shohamy

    Complex learned behaviors must involve the integrated action of distributed brain circuits. While the contributions of individual regions to learning have been extensively investigated, understanding how distributed brain networks orchestrate their activity over the course of learning remains elusive. To address this gap, we used fMRI combined with tools from dynamic network neuroscience to obtain time-resolved descriptions of network coordination during reinforcement learning. We found that learning to associate visual cues with reward involves dynamic changes in network coupling between the striatum and distributed brain regions, including visual, orbitofrontal, and ventromedial prefrontal cortex. Moreover, we found that flexibility in striatal network dynamics correlates with participants’ learning rate and inverse temperature, two parameters derived from reinforcement learning models. Finally, we found that not all forms of learning relate to this circuit: episodic memory, measured in the same participants at the same time, was related to dynamic connectivity in distinct brain networks. These results suggest that dynamic changes in striatal-centered networks provide a mechanism for information integration during reinforcement learning. Significance Statement: Learning from the outcomes of actions–referred to as reinforcement learning –is an essential part of life. The roles of individual brain regions in reinforcement learning have been well characterized in terms of the updating of values for actions or sensory stimuli. Missing from this account, however, is a description of the manner in which different brain areas interact during learning to integrate sensory and value information. Here we characterize flexible striatal-cortical network dynamics that relate to reinforcement learning behavior.


  • Open Access

    Transfer of Learning Relates to Intrinsic Connectivity between Hippocampus, Ventromedial Prefrontal Cortex, and Large-Scale Networks

    by Raphael T. Gerraty, Juliet Y. Davidow, G. Elliott Wimmer, Itamar Kahn, Daphna Shohamy

    An important aspect of adaptive learning is the ability to flexibly use past experiences to guide new decisions. When facing a new decision, some people automatically leverage previously learned associations, while others do not. This variability in transfer of learning across individuals has been demonstrated repeatedly and has important implications for understanding adaptive behavior, yet the source of these individual differences remains poorly understood. In particular, it is unknown why such variability in transfer emerges even among homogeneous groups of young healthy participants who do not vary on other learning-related measures. Here we hypothesized that individual differences in the transfer of learning could be related to relatively stable differences in intrinsic brain connectivity, which could constrain how individuals learn. To test this, we obtained a behavioral measure of memory-based transfer outside of the scanner and on a separate day acquired resting-state functional MRI images in 42 participants. We then analyzed connectivity across independent component analysis-derived brain networks during rest, and tested whether intrinsic connectivity in learning-related networks was associated with transfer. We found that individual differences in transfer were related to intrinsic connectivity between the hippocampus and the ventromedial prefrontal cortex, and between these regions and large-scale functional brain networks. Together, the findings demonstrate a novel role for intrinsic brain dynamics in flexible learning-guided behavior, both within a set of functionally specific regions known to be important for learning, as well as between these regions and the default and frontoparietal networks, which are thought to serve more general cognitive functions.