YPYaozong Pan
Yaozong Pan
*This is an algorithmically generated author record based on publicly available information.
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Yaozong Pan has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- Haitao Yang
Space Engineering University, Beijing 101416, China
1 shared publication - Jian Zhang
Space Engineering University, 81 Road, Huairou District, Beijing 101400, China
1 shared publication - Ruili Wang
Massey University
1 shared publication - Yuqiang Fang
Space Engineering University, Beijing, China
1 shared publication
Latest Publications
- Open Access
- 4 April 2019
- Published by MDPI AG in Applied Sciences
- Vol. 9 (7) , 1430
Multi-Agent Planning under Uncertainty with Monte Carlo Q-Value Function
Decentralized partially observable Markov decision processes (Dec-POMDPs) are general multi-agent models for planning under uncertainty, but are intractable to solve. Doubly exponential growth of the search space as the horizon increases makes a brute-force search impossible. Heuristic methods can guide the search towards the right direction quickly and have been successful in different domains. In this paper, we propose a new Q-value function representation—Monte Carlo Q-value function , which is proved to be an upper bound of the optimal Q-value function . We introduce two Monte Carlo tree search enhancements—heavy playout for a simulation policy and adaptive samples—to speed up computation of . Then, we present a clustering and expansion with Monte-Carlo algorithm (CEMC)—an offline planning algorithm using as Q-value function, which is based on the generalized multi-agent A* with incremental clustering and expansion (GMAA*-ICE or ICE). CEMC calculates Q-value functions as required, without computing and storing all Q-value functions. An extended policy pruning strategy is used in CEMC. Finally, we present empirical results demonstrating that CEMC outperforms the best heuristic algorithm with a compact Q-value presentation in term of runtime for the same horizon, and has less memory usage for larger problems.- 4 April 2019
- Published by MDPI AG in Applied Sciences
- Vol. 9 (7) , 1430
Open AccessMulti-Agent Planning under Uncertainty with Monte Carlo Q-Value Function
Decentralized partially observable Markov decision processes (Dec-POMDPs) are general multi-agent models for planning under uncertainty, but are intractable to solve. Doubly exponential growth of the search space as the horizon increases makes a brute-force search impossible. Heuristic methods can guide the search towards the right direction quickly and have been successful in different domains. In this paper, we propose a new Q-value function representation—Monte Carlo Q-value function , which is proved to be an upper bound of the optimal Q-value function . We introduce two Monte Carlo tree search enhancements—heavy playout for a simulation policy and adaptive samples—to speed up computation of . Then, we present a clustering and expansion with Monte-Carlo algorithm (CEMC)—an offline planning algorithm using as Q-value function, which is based on the generalized multi-agent A* with incremental clustering and expansion (GMAA*-ICE or ICE). CEMC calculates Q-value functions as required, without computing and storing all Q-value functions. An extended policy pruning strategy is used in CEMC. Finally, we present empirical results demonstrating that CEMC outperforms the best heuristic algorithm with a compact Q-value presentation in term of runtime for the same horizon, and has less memory usage for larger problems.