TWTing Han Wei

Ting Han Wei

*This is an algorithmically generated author record based on publicly available information.

Ting Han Wei has a total of 20 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Towards Combining On-Off-Policy Methods for Real-World Applications

    by Kai-Chun Hu, Chen-Huan Pi, Ting Han, Wei ‡3, I-Chen Wu, Cheng Stone, Yi-Wei ¶2, Wei-Yuan Ye

    In this paper, we point out a fundamental property of the objective in reinforcement learning, with which we can reformulate the policy gradient objective into a perceptron-like loss function, removing the need to distinguish between on and off policy training. Namely, we posit that it is sufficient to only update a policy $\pi$ for cases that satisfy the condition $A(\frac{\pi}{\mu}-1)\leq0$, where $A$ is the advantage, and $\mu$ is another policy. Furthermore, we show via theoretic derivation that a perceptron-like loss function matches the clipped surrogate objective for PPO. With our new formulation, the policies $\pi$ and $\mu$ can be arbitrarily apart in theory, effectively enabling off-policy training. To examine our derivations, we can combine the on-policy PPO clipped surrogate (which we show to be equivalent with one instance of the new reformation) with the off-policy IMPALA method. We first verify the combined method on the OpenAI Gym pendulum toy problem. Next, we use our method to train a quadrotor position controller in a simulator. Our trained policy is efficient and lightweight enough to perform in a low cost micro-controller at a minimum update rate of 500 Hz. For the quadrotor, we show two experiments to verify our method and demonstrate performance: 1) hovering at a fixed position, and 2) tracking along a specific trajectory. In preliminary trials, we are also able to apply the method to a real-world quadrotor.

  • Comparison Training for Computer Chinese Chess

    by Jr-Chang Chen, Wen-Jie Tseng, I-Chen Wu, Ting-Han Wei

    This paper describes the application of a modified comparison training for automatic feature weight tuning. The final objective is to improve the evaluation functions used in Chinese chess programs. First, we apply n-tuple networks to extract features. N-tuple networks require very little expert knowledge through its large numbers of features, while simultaneously allowing easy access. Second, we propose a modified comparison training into which tapered eval is incorporated. Experiments show that with the same features and the same Chinese chess program, the automatically tuned feature weights achieved a win rate of 86.58% against the hand-tuned ones. The above trained version was then improved by adding additional features, most importantly n-tuple features. This improved version achieved a win rate of 81.65% against the trained version without additional features.


  • Multilabeled Value Networks for Computer Go

    by Ti-Rong Wu, I-Chen Wu, Guan-Wun Chen, Ting-Han Wei, Hung-Chun Wu, Tung-Yi Lai, Li-Cheng Lan

    This paper proposes a new approach to a novel value network architecture for the game Go, called a multi-labelled (ML) value network. In the ML value network, different values (win rates) are trained simultaneously for different settings of komi, a compensation given to balance the initiative of playing first. The ML value network has three advantages, (a) it outputs values for different komi, (b) it supports dynamic komi, and (c) it lowers the mean squared error (MSE). This paper also proposes a new dynamic komi method to improve game-playing strength. This paper also performs experiments to demonstrate the merits of the architecture. First, the MSE of the ML value network is generally lower than the value network alone. Second, the program based on the ML value network wins by a rate of 67.6% against the program based on the value network alone. Third, the program with the proposed dynamic komi method significantly improves the playing strength over the baseline that does not use dynamic komi, especially for handicap games. To our knowledge, up to date, no handicap games have been played openly by programs using value networks. This paper provides these programs with a useful approach to playing handicap games.


  • Open Access

    Comparison Training for Computer Chinese Chess

    by Wen-Jie Tseng, Jr-Chang Chen, I-Chen Wu, Tinghan Wei

    This paper describes the application of comparison training (CT) for automatic feature weight tuning, with the final objective of improving the evaluation functions used in Chinese chess programs. First, we propose an n-tuple network to extract features, since n-tuple networks require very little expert knowledge through its large numbers of features, while simulta-neously allowing easy access. Second, we propose a novel evalua-tion method that incorporates tapered eval into CT. Experiments show that with the same features and the same Chinese chess program, the automatically tuned comparison training feature weights achieved a win rate of 86.58% against the weights that were hand-tuned. The above trained version was then improved by adding additional features, most importantly n-tuple features. This improved version achieved a win rate of 81.65% against the trained version without additional features.

  • Software Development Framework for Job-Level Algorithms

    by Ting Han Wei, Chao-Chin Liang, I-Chen Wu, Lung-Pin Chen

    Dept. of Computer Science and Information Engineering, National Chiao Tung University, Hsinchu / Taiwan. Email: icwu@cs.nctu.edu.tw ; tinghan.wei@gmail.com Recently, Wu et al. introduced a general distributed computing approach, named Job-Level (JL)


  • Job-Level Algorithms for Connect6 Opening Book Construction

    by Ting Han Wei, I-Chen Wu, Chao-Chin Liang, Bing-Tsung Chiang, Wen-Jie Tseng, Shi-Jim Yen, Chang-Shing Lee

    Dept. of Computer Science and Information Engineering, National Chiao Tung University, Hsinchu / Taiwan. Email: icwu@cs.nctu.edu.tw ; tinghan.wei@gmail.com ; Dept. of CS and Inf. Engineering, National Dong-Hwa University, 1, Scc, Da Hsueh Rd., Shou-F