Random Markov decision processes for sustainable infrastructure systems
Authors:
- 15 April 2014
- journal article
- research article
- Published by Taylor & Francis in Structure and Infrastructure Engineering
- Vol. 11 (5) , 655-667
Abstract
The quantitative assessment of the life-cycle performance of infrastructure systems has seen rapid progress using methods from systems dynamics. Markov chains and Markov decision processes are some of the more useful modelling tools that permit the integration of stochasticity in decisions related to sustainable infrastructure management. In this paper, we explore the effect of uncertainties in the characterisation of these decision models. In particular, we consider the uncertainty in the transition parameters of these models and quantitatively discuss the implication of this kind of uncertainty on the optimal decisions as the solution of these models. We also make the case for the development of rationales that can differentiate between these decision models. In addition, a computationally efficient solution algorithm is introduced for these systems under transition uncertainties. Finally, we demonstrate the sensitivity of the optimal decisions to transition uncertainties and also the computational advantage of the discussed solution algorithm using the problem of optimal maintenance of pavement.Keywords
This publication has 20 references indexed in Scilit:
- Polynomial chaos representation of spatio-temporal random fields from experimental measurementsJournal of Computational Physics, 2009
- Probabilistic models for life‐cycle performance of deteriorating structures: review and future directionsProgress in Structural Engineering and Materials, 2004
- Modeling uncertainty in flow simulations via generalized polynomial chaosJournal of Computational Physics, 2003
- Bounded-parameter Markov decision processesArtificial Intelligence, 2000
- Inspection, Maintenance, and Repair with Partial ObservabilityJournal of Infrastructure Systems, 1995
- Reliability-based bridge design and life cycle management with Markov decision processesStructural Safety, 1994
- Optimal Maintenance Decisions for Pavement ManagementJournal of Transportation Engineering, 1987
- Prior Distributions on Spaces of Probability MeasuresThe Annals of Statistics, 1974
- Information Theory and Statistical MechanicsPhysical Review B, 1957
- Remarks on a Multivariate TransformationThe Annals of Mathematical Statistics, 1952