MDPs with Setwise Continuous Transition Probabilities
Preprint
Abstract
This paper describes the structure of optimal policies for infinite-state Markov Decision Processes with setwise continuous transition probabilities. The action sets may be noncompact. The objective criteria are either the expected total discounted and undiscounted costs or average costs per unit time. The analysis of optimality equations and inequalities is based on the optimal selection theorem for inf-compact functions.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Sufficiency of Deterministic Policies for Atomless Discounted and Uniformly Absorbing MDPs with Multiple CriteriaSIAM Journal on Control and Optimization, 2019
- On the optimality equation for average cost Markov decision processes and its validity for inventory controlAnnals of Operations Research, 2017
- On Convergence of Value Iteration for a Class of Total Cost Markov Decision ProcessesSIAM Journal on Control and Optimization, 2015
- Berge’s theorem for noncompact image setsJournal of Mathematical Analysis and Applications, 2013
- Classical Descriptive Set Theory1995
- Average optimality in dynamic programming on Borel spaces — unbounded costs and controlsSystems & Control Letters, 1991
- Controlled Markov Processes1979
- Measurable selection theorems for optimization problemsmanuscripta mathematica, 1978
- Optimal Plans for Dynamic Programming ProblemsMathematics of Operations Research, 1976
- Measurable Selection and Dynamic ProgrammingMathematics of Operations Research, 1976