Uncertainty and Exploration in a Restless Bandit Problem
- 20 April 2015
- journal article
- research article
- Published by Wiley in Topics in Cognitive Science
- Vol. 7 (2) , 351-367
Abstract
Decision making in noisy and changing environments requires a fine balance between exploiting knowledge about good courses of action and exploring the environment in order to improve upon this knowledge. We present an experiment on a restless bandit task in which participants made repeated choices between options for which the average rewards changed over time. Comparing a number of computational models of participants’ behavior in this task, we find evidence that a substantial number of them balanced exploration and exploitation by considering the probability that an option offers the maximum reward out of all the available options.Keywords
This publication has 21 references indexed in Scilit:
- A Bayesian analysis of human decision-making on bandit problemsJournal of Mathematical Psychology, 2009
- Comparison of Decision Learning Models Using the Generalization Criterion MethodCognitive Science, 2008
- Learning the value of information in an uncertain worldNature Neuroscience, 2007
- Should I stay or should I go? How the human brain manages the trade-off between exploitation and explorationPhilosophical Transactions of the Royal Society B: Biological Sciences, 2007
- Cortical substrates for exploratory decisions in humansNature, 2006
- A contribution of cognitive decision models to clinical assessment: Decomposing performance on the Bechara gambling task.Psychological Assessment, 2002
- Restless bandits: activity allocation in a changing worldJournal of Applied Probability, 1988
- New Results in Linear Filtering and Prediction TheoryJournal of Basic Engineering, 1961
- A New Approach to Linear Filtering and Prediction ProblemsJournal of Basic Engineering, 1960
- ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLESBiometrika, 1933