Shengbo Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Shengbo Wang has a total of 2 co-authors. Here are the top co-authors by number of shared publications:
- Julien Grand-Clément
HEC Paris
1 shared publication - Nian Si
Department of Industrial Engineering and Decision Analytics Hong Kong University of Science and Technology
1 shared publication
Latest Publications
- Open Access
Tractable Robust Markov Decision Processes
In this paper we investigate the tractability of robust Markov Decision Processes (RMDPs) under various structural assumptions on the uncertainty set. Surprisingly, we show that in all generality (i.e. without any assumption on the instantaneous rewards), s-rectangular and sa-rectangular uncertainty sets are the only models of uncertainty that are tractable. Our analysis also shows that existing non-rectangular models, including r-rectangular uncertainty and new generalizations, are only weakly tractable in that they require an additional structural assumption that the instantaneous rewards do not depend on the next state, and in this case they are equivalent to rectangular models, which severely undermines their significance and usefulness. Interestingly, our proof techniques rely on identifying a novel simultaneous solvability property, which we show is at the heart of several important properties of RMDPs, including the existence of stationary optimal policies and dynamic programming-based formulations. The simultaneous solvability property enables a unified approach to studying the tractability of all existing models of uncertainty, rectangular and non-rectangular alike.
Open AccessTractable Robust Markov Decision Processes
In this paper we investigate the tractability of robust Markov Decision Processes (RMDPs) under various structural assumptions on the uncertainty set. Surprisingly, we show that in all generality (i.e. without any assumption on the instantaneous rewards), s-rectangular and sa-rectangular uncertainty sets are the only models of uncertainty that are tractable. Our analysis also shows that existing non-rectangular models, including r-rectangular uncertainty and new generalizations, are only weakly tractable in that they require an additional structural assumption that the instantaneous rewards do not depend on the next state, and in this case they are equivalent to rectangular models, which severely undermines their significance and usefulness. Interestingly, our proof techniques rely on identifying a novel simultaneous solvability property, which we show is at the heart of several important properties of RMDPs, including the existence of stationary optimal policies and dynamic programming-based formulations. The simultaneous solvability property enables a unified approach to studying the tractability of all existing models of uncertainty, rectangular and non-rectangular alike.