Large-scale tabular-form hardware architecture for Q-Learning with delays
Authors:
- 1 August 2007
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15483746,p. 827-830
Abstract
Q-Learning is a popular reinforcement learning algorithm which has been widely used in stochastic control applications. The bottleneck of applying tabular form Q learning in reinforcement learning problems with large scale or high dimensional action sets is the considerable delays caused by action selection and value function updates. In this paper, we present a novel hardware architecture that significantly reduces the delays. Moreover, we formulate the Q learning algorithm in cases of observation and action delays and provide a set of proofs confirming that Q-Learning with such delays converges to the optimal policy.Keywords
This publication has 4 references indexed in Scilit:
- Markov decision processes with delays and asynchronous cost collectionIEEE Transactions on Automatic Control, 2003
- Asynchronous stochastic approximation and Q-learningMachine Learning, 1994
- Closed-loop control with delayed informationACM SIGMETRICS Performance Evaluation Review, 1992
- A Partially Observable Markov Decision Process with Lagged InformationJournal of the Operational Research Society, 1987