Temporal Difference Model Reproduces Anticipatory Neural Activity
Authors:
- 1 April 2001
- journal article
- Published by MIT Press in Neural Computation
- Vol. 13 (4) , 841-862
Abstract
Anticipatory neural activity preceding behaviorally important events has been reported in cortex, striatum, and midbrain dopamine neurons. Whereas dopamine neurons are phasically activated by reward-predictive stimuli, anticipatory activity of cortical and striatal neurons is increased during delay periods before important events. Characteristics of dopa-mine neuron activity resemble those of the prediction error signal of the temporal difference (TD) model of Pavlovian learning (Sutton & Barto, 1990). This study demonstrates that the prediction signal of the TD model reproduces characteristics of cortical and striatal anticipatory neural activity. This finding suggests that tonic anticipatory activities may reflect prediction signals that are involved in the processing of dopamine neuron activity.Keywords
This publication has 41 references indexed in Scilit:
- Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effectsNature Neuroscience, 1999
- Dopamine neurons report an error in the temporal prediction of reward during learningNature Neuroscience, 1998
- A Model of Corticostriatal Plasticity for Learning Oculomotor Associations and SequencesJournal of Cognitive Neuroscience, 1995
- Improving Generalization for Temporal Difference Learning: The Successor RepresentationNeural Computation, 1993
- Long‐term Potentiation in the Striatum is Unmasked by Removing the Voltage‐dependent Magnesium Block of NMDA Receptor ChannelsEuropean Journal of Neuroscience, 1992
- The Updating of the Representation of Visual Space in Parietal Cortex by Intended Eye MovementsScience, 1992
- Altering the synchrony of stimulus trace processes: tests of a neural-network modelBiological Cybernetics, 1991
- Neural dynamics of adaptive timing and temporal discrimination during associative learningNeural Networks, 1989
- Adaptive timing in neural networks: The conditioned responseBiological Cybernetics, 1988
- Premotor cortex of the rhesus monkey: neuronal activity in anticipation of predictable environmental eventsExperimental Brain Research, 1986