Discounted cost exponential semi-Markov decision processes with unbounded transition rates: a service rate control problem with impatient customers
Author:
Open Access
- 18 April 2024
- journal article
- research article
- Published by Cambridge University Press (CUP) in Probability in the Engineering and Informational Sciences
- Vol. 38 (4) , 668-694
Abstract
We focus on exponential semi-Markov decision processes with unbounded transition rates. We first provide several sufficient conditions under which the value iteration procedure converges to the optimal value function and optimal deterministic stationary policies exist. These conditions are also valid for general semi-Markov decision processes possibly with accumulation points. Then, we apply our results to a service rate control problem with impatient customers. The resulting exponential semi-Markov decision process has unbounded transition rates, which makes the well-known uniformization technique inapplicable. We analyze the structure of the optimal policy and the monotonicity of the optimal value function by using the customization technique that was introduced by the author in prior work.Keywords
This publication has 21 references indexed in Scilit:
- Adaptive service rate control of an M/M/1 queue with server breakdownsQueueing Systems, 2024
- MDPs with setwise continuous transition probabilitiesOperations Research Letters, 2021
- Continuous-Time Markov Decision Processes2020
- Customizing exponential semi-Markov decision processes under the discounted cost criterionEuropean Journal of Operational Research, 2018
- Dynamic control of a tandem system with abandonmentsQueueing Systems, 2016
- On structural properties of the value function for an unbounded jump Markov process with an application to a processor sharing retrial queueQueueing Systems, 2013
- Berge’s theorem for noncompact image setsJournal of Mathematical Analysis and Applications, 2013
- Reduction of Discounted Continuous-Time MDPs with Unbounded Jump and Reward Rates to Discrete-Time Total-Reward MDPs2012
- Probability and Stochastics2011
- Constrained Discounted Semi-Markov Decision Processes2002