Ting Han Wei
Research Subjects
Research Interests
Ting Han Wei has a total of 20 co-authors. Here are the top co-authors by number of shared publications:
- I-Chen Wu
National yang Ming Chiao Tung University
6 shared publications Chao-Chin Liang
2 shared publications- Jr-Chang Chen
National Taipei University
2 shared publications - Wen-Jie Tseng
Department of Computer Science, National Yang Ming Chiao Tung University
2 shared publications Bing-Tsung Chiang
1 shared publication- Chang-Shing Lee
Department of Computer Science and Information Engineering, National University of Tainan, Tainan, Taiwan
1 shared publication
Latest Publications
- Open Access
Towards Combining On-Off-Policy Methods for Real-World Applications
In this paper, we point out a fundamental property of the objective in reinforcement learning, with which we can reformulate the policy gradient objective into a perceptron-like loss function, removing the need to distinguish between on and off policy training. Namely, we posit that it is sufficient to only update a policy $\pi$ for cases that satisfy the condition $A(\frac{\pi}{\mu}-1)\leq0$, where $A$ is the advantage, and $\mu$ is another policy. Furthermore, we show via theoretic derivation that a perceptron-like loss function matches the clipped surrogate objective for PPO. With our new formulation, the policies $\pi$ and $\mu$ can be arbitrarily apart in theory, effectively enabling off-policy training. To examine our derivations, we can combine the on-policy PPO clipped surrogate (which we show to be equivalent with one instance of the new reformation) with the off-policy IMPALA method. We first verify the combined method on the OpenAI Gym pendulum toy problem. Next, we use our method to train a quadrotor position controller in a simulator. Our trained policy is efficient and lightweight enough to perform in a low cost micro-controller at a minimum update rate of 500 Hz. For the quadrotor, we show two experiments to verify our method and demonstrate performance: 1) hovering at a fixed position, and 2) tracking along a specific trajectory. In preliminary trials, we are also able to apply the method to a real-world quadrotor.Open AccessTowards Combining On-Off-Policy Methods for Real-World Applications
In this paper, we point out a fundamental property of the objective in reinforcement learning, with which we can reformulate the policy gradient objective into a perceptron-like loss function, removing the need to distinguish between on and off policy training. Namely, we posit that it is sufficient to only update a policy $\pi$ for cases that satisfy the condition $A(\frac{\pi}{\mu}-1)\leq0$, where $A$ is the advantage, and $\mu$ is another policy. Furthermore, we show via theoretic derivation that a perceptron-like loss function matches the clipped surrogate objective for PPO. With our new formulation, the policies $\pi$ and $\mu$ can be arbitrarily apart in theory, effectively enabling off-policy training. To examine our derivations, we can combine the on-policy PPO clipped surrogate (which we show to be equivalent with one instance of the new reformation) with the off-policy IMPALA method. We first verify the combined method on the OpenAI Gym pendulum toy problem. Next, we use our method to train a quadrotor position controller in a simulator. Our trained policy is efficient and lightweight enough to perform in a low cost micro-controller at a minimum update rate of 500 Hz. For the quadrotor, we show two experiments to verify our method and demonstrate performance: 1) hovering at a fixed position, and 2) tracking along a specific trajectory. In preliminary trials, we are also able to apply the method to a real-world quadrotor. - 16 January 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Games
- Vol. 12 (2) , 169-176
Comparison Training for Computer Chinese Chess
This paper describes the application of a modified comparison training for automatic feature weight tuning. The final objective is to improve the evaluation functions used in Chinese chess programs. First, we apply n-tuple networks to extract features. N-tuple networks require very little expert knowledge through its large numbers of features, while simultaneously allowing easy access. Second, we propose a modified comparison training into which tapered eval is incorporated. Experiments show that with the same features and the same Chinese chess program, the automatically tuned feature weights achieved a win rate of 86.58% against the hand-tuned ones. The above trained version was then improved by adding additional features, most importantly n-tuple features. This improved version achieved a win rate of 81.65% against the trained version without additional features.
- 16 January 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Games
- Vol. 12 (2) , 169-176
Comparison Training for Computer Chinese Chess
This paper describes the application of a modified comparison training for automatic feature weight tuning. The final objective is to improve the evaluation functions used in Chinese chess programs. First, we apply n-tuple networks to extract features. N-tuple networks require very little expert knowledge through its large numbers of features, while simultaneously allowing easy access. Second, we propose a modified comparison training into which tapered eval is incorporated. Experiments show that with the same features and the same Chinese chess program, the automatically tuned feature weights achieved a win rate of 86.58% against the hand-tuned ones. The above trained version was then improved by adding additional features, most importantly n-tuple features. This improved version achieved a win rate of 81.65% against the trained version without additional features.
- 4 July 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Games
- Vol. 10 (4) , 378-389
Multilabeled Value Networks for Computer Go
This paper proposes a new approach to a novel value network architecture for the game Go, called a multi-labelled (ML) value network. In the ML value network, different values (win rates) are trained simultaneously for different settings of komi, a compensation given to balance the initiative of playing first. The ML value network has three advantages, (a) it outputs values for different komi, (b) it supports dynamic komi, and (c) it lowers the mean squared error (MSE). This paper also proposes a new dynamic komi method to improve game-playing strength. This paper also performs experiments to demonstrate the merits of the architecture. First, the MSE of the ML value network is generally lower than the value network alone. Second, the program based on the ML value network wins by a rate of 67.6% against the program based on the value network alone. Third, the program with the proposed dynamic komi method significantly improves the playing strength over the baseline that does not use dynamic komi, especially for handicap games. To our knowledge, up to date, no handicap games have been played openly by programs using value networks. This paper provides these programs with a useful approach to playing handicap games.
- 4 July 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Games
- Vol. 10 (4) , 378-389
Multilabeled Value Networks for Computer Go
This paper proposes a new approach to a novel value network architecture for the game Go, called a multi-labelled (ML) value network. In the ML value network, different values (win rates) are trained simultaneously for different settings of komi, a compensation given to balance the initiative of playing first. The ML value network has three advantages, (a) it outputs values for different komi, (b) it supports dynamic komi, and (c) it lowers the mean squared error (MSE). This paper also proposes a new dynamic komi method to improve game-playing strength. This paper also performs experiments to demonstrate the merits of the architecture. First, the MSE of the ML value network is generally lower than the value network alone. Second, the program based on the ML value network wins by a rate of 67.6% against the program based on the value network alone. Third, the program with the proposed dynamic komi method significantly improves the playing strength over the baseline that does not use dynamic komi, especially for handicap games. To our knowledge, up to date, no handicap games have been played openly by programs using value networks. This paper provides these programs with a useful approach to playing handicap games.
- Open Access
Comparison Training for Computer Chinese Chess
This paper describes the application of comparison training (CT) for automatic feature weight tuning, with the final objective of improving the evaluation functions used in Chinese chess programs. First, we propose an n-tuple network to extract features, since n-tuple networks require very little expert knowledge through its large numbers of features, while simulta-neously allowing easy access. Second, we propose a novel evalua-tion method that incorporates tapered eval into CT. Experiments show that with the same features and the same Chinese chess program, the automatically tuned comparison training feature weights achieved a win rate of 86.58% against the weights that were hand-tuned. The above trained version was then improved by adding additional features, most importantly n-tuple features. This improved version achieved a win rate of 81.65% against the trained version without additional features.
Open AccessComparison Training for Computer Chinese Chess
This paper describes the application of comparison training (CT) for automatic feature weight tuning, with the final objective of improving the evaluation functions used in Chinese chess programs. First, we propose an n-tuple network to extract features, since n-tuple networks require very little expert knowledge through its large numbers of features, while simulta-neously allowing easy access. Second, we propose a novel evalua-tion method that incorporates tapered eval into CT. Experiments show that with the same features and the same Chinese chess program, the automatically tuned comparison training feature weights achieved a win rate of 86.58% against the weights that were hand-tuned. The above trained version was then improved by adding additional features, most importantly n-tuple features. This improved version achieved a win rate of 81.65% against the trained version without additional features.
- 1 September 2015
- Published by SAGE Publications in ICGA Journal: The Journal of the Computer Games Community
- Vol. 38 (3) , 131-148
Software Development Framework for Job-Level Algorithms
Dept. of Computer Science and Information Engineering, National Chiao Tung University, Hsinchu / Taiwan. Email: icwu@cs.nctu.edu.tw ; tinghan.wei@gmail.com Recently, Wu et al. introduced a general distributed computing approach, named Job-Level (JL)
- 1 September 2015
- Published by SAGE Publications in ICGA Journal: The Journal of the Computer Games Community
- Vol. 38 (3) , 131-148
Software Development Framework for Job-Level Algorithms
Dept. of Computer Science and Information Engineering, National Chiao Tung University, Hsinchu / Taiwan. Email: icwu@cs.nctu.edu.tw ; tinghan.wei@gmail.com Recently, Wu et al. introduced a general distributed computing approach, named Job-Level (JL)
- 1 September 2015
- Published by SAGE Publications in ICGA Journal: The Journal of the Computer Games Community
- Vol. 38 (3) , 165-179
Job-Level Algorithms for Connect6 Opening Book Construction
Dept. of Computer Science and Information Engineering, National Chiao Tung University, Hsinchu / Taiwan. Email: icwu@cs.nctu.edu.tw ; tinghan.wei@gmail.com ; Dept. of CS and Inf. Engineering, National Dong-Hwa University, 1, Scc, Da Hsueh Rd., Shou-F
- 1 September 2015
- Published by SAGE Publications in ICGA Journal: The Journal of the Computer Games Community
- Vol. 38 (3) , 165-179
Job-Level Algorithms for Connect6 Opening Book Construction
Dept. of Computer Science and Information Engineering, National Chiao Tung University, Hsinchu / Taiwan. Email: icwu@cs.nctu.edu.tw ; tinghan.wei@gmail.com ; Dept. of CS and Inf. Engineering, National Dong-Hwa University, 1, Scc, Da Hsueh Rd., Shou-F