Jian Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jian Zhang has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Chunhui Yuan
Space Engineering University, Beijing 101416, China
1 shared publication - Haitao Yang
Space Engineering University, Beijing 101416, China
1 shared publication - Haitao Yang
Space Engineering University, 81 Road, Huairou District, Beijing 101400, China
1 shared publication - Ruili Wang
Massey University
1 shared publication - Yaozong Pan
Shandong Provincial Key Laboratory, Naval Aviation University, Yantai 264001, China
1 shared publication - Yaozong Pan
Space Engineering University, Beijing 101416, China
1 shared publication
Latest Publications
- Open Access
Supervised Reinforcement Learning via Value Function
Using expert samples to improve the performance of reinforcement learning (RL) algorithms has become one of the focuses of research nowadays. However, in different application scenarios, it is hard to guarantee both the quantity and quality of expert samples, which prohibits the practical application and performance of such algorithms. In this paper, a novel RL decision optimization method is proposed. The proposed method is capable of reducing the dependence on expert samples via incorporating the decision-making evaluation mechanism. By introducing supervised learning (SL), our method optimizes the decision making of the RL algorithm by using demonstrations or expert samples. Experiments are conducted in Pendulum and Puckworld scenarios to test the proposed method, and we use representative algorithms such as deep Q-network (DQN) and Double DQN (DDQN) as benchmarks. The results demonstrate that the method adopted in this paper can effectively improve the decision-making performance of agents even when the expert samples are not available.
Open AccessSupervised Reinforcement Learning via Value Function
Using expert samples to improve the performance of reinforcement learning (RL) algorithms has become one of the focuses of research nowadays. However, in different application scenarios, it is hard to guarantee both the quantity and quality of expert samples, which prohibits the practical application and performance of such algorithms. In this paper, a novel RL decision optimization method is proposed. The proposed method is capable of reducing the dependence on expert samples via incorporating the decision-making evaluation mechanism. By introducing supervised learning (SL), our method optimizes the decision making of the RL algorithm by using demonstrations or expert samples. Experiments are conducted in Pendulum and Puckworld scenarios to test the proposed method, and we use representative algorithms such as deep Q-network (DQN) and Double DQN (DDQN) as benchmarks. The results demonstrate that the method adopted in this paper can effectively improve the decision-making performance of agents even when the expert samples are not available.
- Open Access
- 4 April 2019
- Published by MDPI AG in Applied Sciences
- Vol. 9 (7) , 1430
Multi-Agent Planning under Uncertainty with Monte Carlo Q-Value Function
Decentralized partially observable Markov decision processes (Dec-POMDPs) are general multi-agent models for planning under uncertainty, but are intractable to solve. Doubly exponential growth of the search space as the horizon increases makes a brute-force search impossible. Heuristic methods can guide the search towards the right direction quickly and have been successful in different domains. In this paper, we propose a new Q-value function representation—Monte Carlo Q-value function , which is proved to be an upper bound of the optimal Q-value function . We introduce two Monte Carlo tree search enhancements—heavy playout for a simulation policy and adaptive samples—to speed up computation of . Then, we present a clustering and expansion with Monte-Carlo algorithm (CEMC)—an offline planning algorithm using as Q-value function, which is based on the generalized multi-agent A* with incremental clustering and expansion (GMAA*-ICE or ICE). CEMC calculates Q-value functions as required, without computing and storing all Q-value functions. An extended policy pruning strategy is used in CEMC. Finally, we present empirical results demonstrating that CEMC outperforms the best heuristic algorithm with a compact Q-value presentation in term of runtime for the same horizon, and has less memory usage for larger problems.- 4 April 2019
- Published by MDPI AG in Applied Sciences
- Vol. 9 (7) , 1430
Open AccessMulti-Agent Planning under Uncertainty with Monte Carlo Q-Value Function
Decentralized partially observable Markov decision processes (Dec-POMDPs) are general multi-agent models for planning under uncertainty, but are intractable to solve. Doubly exponential growth of the search space as the horizon increases makes a brute-force search impossible. Heuristic methods can guide the search towards the right direction quickly and have been successful in different domains. In this paper, we propose a new Q-value function representation—Monte Carlo Q-value function , which is proved to be an upper bound of the optimal Q-value function . We introduce two Monte Carlo tree search enhancements—heavy playout for a simulation policy and adaptive samples—to speed up computation of . Then, we present a clustering and expansion with Monte-Carlo algorithm (CEMC)—an offline planning algorithm using as Q-value function, which is based on the generalized multi-agent A* with incremental clustering and expansion (GMAA*-ICE or ICE). CEMC calculates Q-value functions as required, without computing and storing all Q-value functions. An extended policy pruning strategy is used in CEMC. Finally, we present empirical results demonstrating that CEMC outperforms the best heuristic algorithm with a compact Q-value presentation in term of runtime for the same horizon, and has less memory usage for larger problems.