Q-Learning for Risk-Sensitive Control
Author:
- 1 May 2002
- journal article
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Mathematics of Operations Research
- Vol. 27 (2) , 294-311
Abstract
We propose for risk-sensitive control of finite Markov chains a counterpart of the popular Q-learning algorithm for classical Markov decision processes. The algorithm is shown to converge with probability one to the desired solution. The proof technique is an adaptation of the o.d.e. approach for the analysis of stochastic approximation algorithms, with most of the work involved used for the analysis of the specific o.d.e.s that arise.Keywords
This publication has 25 references indexed in Scilit:
- Multiplicative ergodicity and large deviations for an irreducible Markov chainStochastic Processes and their Applications, 2000
- A LEARNING ALGORITHM FOR DISCRETE-TIME STOCHASTIC CONTROLProbability in the Engineering and Informational Sciences, 2000
- Risk sensitive asset management with transaction costsFinance and Stochastics, 2000
- An analysis of temporal-difference learning with function approximationIEEE Transactions on Automatic Control, 1997
- Stochastic approximation with two time scalesSystems & Control Letters, 1997
- Connections between stochastic control and dynamic gamesMathematics of Control, Signals, and Systems, 1996
- Risk sensitive control of Markov processes in countable state spaceSystems & Control Letters, 1996
- On the Convergence of Stochastic Iterative Dynamic Programming AlgorithmsNeural Computation, 1994
- Asynchronous stochastic approximation and Q-learningMachine Learning, 1994
- Analysis of recursive stochastic algorithmsIEEE Transactions on Automatic Control, 1977