Encoding of Both Positive and Negative Reward Prediction Errors by Neurons of the Primate Lateral Prefrontal Cortex and Caudate Nucleus
Authors:
- Wael F. AsaadWAWael F. Asaad
- Emad N. EskandarEEEmad N. Eskandar
Open Access
- 7 December 2011
- journal article
- Published by Society for Neuroscience in Journal of Neuroscience
- Vol. 31 (49) , 17772-17787
Abstract
Learning can be motivated by unanticipated success or unexpected failure. The former encourages us to repeat an action or activity, whereas the latter leads us to find an alternative strategy. Understanding the neural representation of these unexpected events is therefore critical to elucidate learning-related circuits. We examined the activity of neurons in the lateral prefrontal cortex (PFC) and caudate nucleus of monkeys as they performed a trial-and-error learning task. Unexpected outcomes were widely represented in both structures, and neurons driven by unexpectedly negative outcomes were as frequent as those activated by unexpectedly positive outcomes. Moreover, both positive and negative reward prediction errors (RPEs) were represented primarily by increases in firing rate, unlike the manner in which dopamine neurons have been observed to reflect these values. Interestingly, positive RPEs tended to appear with shorter latency than negative RPEs, perhaps reflecting the mechanism of their generation. Last, in the PFC but not the caudate, trial-by-trial variations in outcome-related activity were linked to the animals' subsequent behavioral decisions. More broadly, the robustness of RPE signaling by these neurons suggests that actor-critic models of reinforcement learning in which the PFC and particularly the caudate are considered primarily to be “actors” rather than “critics,” should be reconsidered to include a prominent evaluative role for these structures.Keywords
This publication has 58 references indexed in Scilit:
- Surprise Signals in Anterior Cingulate Cortex: Neuronal Encoding of Unsigned Reward Prediction Errors Driving Adjustment in BehaviorThe Journal of Neuroscience, 2011
- Ventral Striatal Neurons Encode the Value of the Chosen Action in Rats Deciding between Differently Delayed or Sized RewardsThe Journal of Neuroscience, 2009
- Neuronal Correlates of Instrumental Learning in the Dorsal StriatumJournal of Neurophysiology, 2009
- Learning Substrates in the Primate Prefrontal Cortex and Striatum: Sustained Activity Related to Successful ActionsNeuron, 2009
- Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variablesEuropean Journal of Neuroscience, 2009
- Two types of dopamine neuron distinctly convey positive and negative motivational signalsNature, 2009
- A flexible software tool for temporally-precise behavioral control in MatlabJournal of Neuroscience Methods, 2008
- Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learningProceedings of the National Academy of Sciences, 2007
- Differential Encoding of Losses and Gains in the Human StriatumJournal of Neuroscience, 2007
- A Neural Circuit Model of Flexible Sensorimotor Mapping: Learning and Forgetting on Multiple TimescalesNeuron, 2007