Abstract
Tibetan word segmentation and POS tagging are the primary tasks of Tibetan natural language processing. Most of existing methods of Tibetan word segmentation and POS tagging are based on rules and statistics, which need manual construction of features. In addition, the joint mode has shown stronger capabilities for word segmentation and POS tagging and have received great interests. In this paper, we propose Bi-LSTM+IDCNN+CRF structures, a simple yet effective end-to-end neural network model, for joint Tibetan word segmentation and POS tagging. We conduct step-by-step and joint experiments on the Tibetan datasets. The results demonstrate that the performance of the Bi-LSTM+IDCNN+CRF model is the best regardless of the step-by-step or joint mode. We obtain state-of-the-art performance in the joint tagging mode. The F1 score of the word segmentation task reached 92.31%, and the F1 score of the POS tagging task reached 81.26%.
Funding Information
  • National Key R&D Program of China (2020YFC0832500)
  • Ministry of Education - China Mobile Research Foundation (MCM20170206)
  • Fundamental Research Funds for the Central Universities (lzujbky-2019-kb51 and lzujbky-2018-k12)
  • National Natural Science Foundation of China (61402210)
  • Major National Project of High Resolution Earth Observation System (30-Y20A34-9010-15/17)
  • State Grid Corporation of China Science and Technology Project (SGGSKY00WYJS2000062)
  • Program for New Century Excellent Talents in University (NCET-12-0250)
  • Strategic Priority Research Program of the Chinese Academy of Sciences (XDA03030100)
  • Google Research Awards and Google Faculty Award, Science and Technology Plan of Qinghai Province (2020-GX-164)