Dopamine Ramps Are a Consequence of Reward Prediction Errors
- 1 March 2014
- journal article
- Published by MIT Press in Neural Computation
- Vol. 26 (3) , 467-471
Abstract
Temporal difference learning models of dopamine assert that phasic levels of dopamine encode a reward prediction error. However, this hypothesis has been challenged by recent observations of gradually ramping stratal dopamine levels as a goal is approached. This note describes conditions under which temporal difference learning models predict dopamine ramping. The key idea is representational: a quadratic transformation of proximity to the goal implies approximately linear ramping, as observed experimentally.Keywords
This publication has 14 references indexed in Scilit:
- Prolonged dopamine signalling in striatum signals proximity and value of distant rewardsNature, 2013
- Grid Cells, Place Cells, and Geodesic Generalization for Spatial Reinforcement LearningPLOS Computational Biology, 2011
- Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesisProceedings of the National Academy of Sciences, 2011
- Temporal-Difference Reinforcement Learning with Distributed RepresentationsPLOS ONE, 2009
- Stimulus Representation and the Timing of Reward-Prediction Errors in Models of the Dopamine SystemNeural Computation, 2008
- Representation and Timing in Theories of the Dopamine SystemNeural Computation, 2006
- A model of hippocampally dependent navigation, using the temporal difference learning ruleHippocampus, 2000
- Error is proportional to distance measured by honeybees: Weber’s law in the odometerAnimal Cognition, 1999
- Linear Least-Squares algorithms for temporal difference learningMachine Learning, 1996
- More psychophysics of the pigeon's use of landmarksJournal of Comparative Physiology A, 1990