MDPs with Setwise Continuous Transition Probabilities
Preprint
Abstract
This paper describes the structure of optimal policies for infinite-state Markov Decision Processes with setwise continuous transition probabilities. The action sets may be noncompact. The objective criteria are either the expected total discounted and undiscounted costs or average costs per unit time. The analysis of optimality equations and inequalities is based on the optimal selection theorem for inf-compact functions introduced in this paper.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Sufficiency of Deterministic Policies for Atomless Discounted and Uniformly Absorbing MDPs with Multiple CriteriaSIAM Journal on Control and Optimization, 2019
- On the optimality equation for average cost Markov decision processes and its validity for inventory controlAnnals of Operations Research, 2017
- On Convergence of Value Iteration for a Class of Total Cost Markov Decision ProcessesSIAM Journal on Control and Optimization, 2015
- Berge’s theorem for noncompact image setsJournal of Mathematical Analysis and Applications, 2013
- Classical Descriptive Set Theory1995
- Average optimality in dynamic programming on Borel spaces — unbounded costs and controlsSystems & Control Letters, 1991
- Stationary policies and Markov policies in Borel dynamic programmingProbability Theory and Related Fields, 1987
- Controlled Markov Processes1979
- Measurable selection theorems for optimization problemsmanuscripta mathematica, 1978
- Negative Dynamic ProgrammingThe Annals of Mathematical Statistics, 1966