A Comparison of Monte Carlo Tree Search and Mathematical Optimization for Large Scale Dynamic Resource Allocation
Preprint
Abstract
Dynamic resource allocation (DRA) problems are an important class of dynamic stochastic optimization problems that arise in a variety of important real-world applications. DRA problems are notoriously difficult to solve to optimality since they frequently combine stochastic elements with intractably large state and action spaces. Although the artificial intelligence and operations research communities have independently proposed two successful frameworks for solving dynamic stochastic optimization problems---Monte Carlo tree search (MCTS) and mathematical optimization (MO), respectively---the relative merits of these two approaches are not well understood. In this paper, we adapt both MCTS and MO to a problem inspired by tactical wildfire and management and undertake an extensive computational study comparing the two methods on large scale instances in terms of both the state and the action spaces. We show that both methods are able to greatly improve on a baseline, problem-specific heuristic. On smaller instances, the MCTS and MO approaches perform comparably, but the MO approach outperforms MCTS as the size of the problem increases for a fixed computational budget.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Dynamic resource allocation: A flexible and tractable modeling frameworkEuropean Journal of Operational Research, 2014
- Monte-Carlo tree search and rapid action value estimation in computer GoArtificial Intelligence, 2011
- Computer poker: A reviewArtificial Intelligence, 2011
- Approximate dynamic programming for dynamic capacity allocation with multiple priority levelsIIE Transactions, 2010
- Monte Carlo tree search in KriegspielArtificial Intelligence, 2010
- A stochastic forest fire growth modelEnvironmental and Ecological Statistics, 2008
- Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search2007
- Analysing initial attack on wildland fires using stochastic simulation*International Journal of Wildland Fire, 2006
- Finite-time Analysis of the Multiarmed Bandit ProblemMachine Learning, 2002
- A genetic algorithm tutorialStatistics and Computing, 1994