Reinforcement learning, conditioning, and the brain: Successes and challenges
Author:
Open Access
- 1 December 2009
- journal article
- review article
- Published by Springer Nature in Cognitive, Affective, & Behavioral Neuroscience
- Vol. 9 (4) , 343-364
Abstract
The field of reinforcement learning has greatly influenced the neuroscientific study of conditioning. This article provides an introduction to reinforcement learning followed by an examination of the successes and challenges using reinforcement learning to understand the neural bases of conditioning. Successes reviewed include (1) the mapping of positive and negative prediction errors to the firing of dopamine neurons and neurons in the lateral habenula, respectively; (2) the mapping of model-based and model-free reinforcement learning to associative and sensorimotor cortico-basal ganglia-thalamo-cortical circuits, respectively; and (3) the mapping of actor and critic to the dorsal and ventral striatum, respectively. Challenges reviewed consist of several behavioral and neural findings that are at odds with standard reinforcement-learning models, including, among others, evidence for hyperbolic discounting and adaptive coding. The article suggests ways of reconciling reinforcement-learning models with many of the challenging findings, and highlights the need for further theoretical developments where necessary. Additional information related to this study may be downloaded from http://cabn.psychonomic-journals.org/content/supplemental.Keywords
This publication has 182 references indexed in Scilit:
- Hierarchically organized behavior and its neural foundations: A reinforcement learning perspectiveCognition, 2009
- Two types of dopamine neuron distinctly convey positive and negative motivational signalsNature, 2009
- Phasic excitation of dopamine neurons in ventral VTA by noxious stimuliProceedings of the National Academy of Sciences, 2009
- Representation of negative motivational value in the primate lateral habenulaNature Neuroscience, 2008
- Prefrontal Coding of Temporally Discounted Values during Intertemporal ChoiceNeuron, 2008
- Nonlinear neurobiological probability weighting functions for aversive outcomesNeuroImage, 2008
- The neural correlates of subjective value during intertemporal choiceNature Neuroscience, 2007
- Dopamine-dependent prediction errors underpin reward-seeking behaviour in humansNature, 2006
- Orbitofrontal Cortex, Associative Learning, and ExpectanciesNeuron, 2005
- A Neural Substrate of Prediction and RewardScience, 1997