Dopaminergic prediction errors in the ventral tegmental area reflect a multithreaded predictive model
- 20 April 2023
- journal article
- research article
- Published by Springer Nature in Nature Neuroscience
- Vol. 26 (5) , 830-839
Abstract
Dopamine neuron activity is tied to the prediction error in temporal difference reinforcement learning models. These models make significant simplifying assumptions, particularly with regard to the structure of the predictions fed into the dopamine neurons, which consist of a single chain of timepoint states. Although this predictive structure can explain error signals observed in many studies, it cannot cope with settings where subjects might infer multiple independent events and outcomes. In the present study, we recorded dopamine neurons in the ventral tegmental area in such a setting to test the validity of the single-stream assumption. Rats were trained in an odor-based choice task, in which the timing and identity of one of several rewards delivered in each trial changed across trial blocks. This design revealed an error signaling pattern that requires the dopamine neurons to access and update multiple independent predictive streams reflecting the subject’s belief about timing and potentially unique identities of expected rewards.Keywords
Funding Information
- U.S. Department of Health & Human Services | NIH | National Institute on Drug Abuse (ZIA-DA000587)
This publication has 39 references indexed in Scilit:
- Effects of Prefrontal Cortical Inactivation on Neural Activity in the Ventral Tegmental AreaJournal of Neuroscience, 2013
- Evaluating the TD model of classical conditioningLearning & Behavior, 2012
- Neuron-type-specific signals for reward and punishment in the ventral tegmental areaNature, 2012
- Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortexNature Neuroscience, 2011
- Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesisProceedings of the National Academy of Sciences of the United States of America, 2011
- States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement LearningNeuron, 2010
- The temporal precision of reward prediction in dopamine neuronsNature Neuroscience, 2008
- Representation and Timing in Theories of the Dopamine SystemNeural Computation, 2006
- Planning and acting in partially observable stochastic domainsArtificial Intelligence, 1998
- A Neural Substrate of Prediction and RewardScience, 1997