Named entity recognition in Bengali using system combination
Authors:
- 5 September 2014
- journal article
- Published by John Benjamins Publishing Company
- Vol. 37 (1) , 1-22
Abstract
This paper reports a voted Named Entity Recognition (NER) system that exploits appropriate unlabeled data. Initially, we develop NER systems using the supervised machine learning algorithms such as Maximum Entropy (ME), Conditional Random Field (CRF) and Support Vector Machine (SVM). Each of these models makes use of the language independent features in the form of different contextual and orthographic word-level features along with the language dependent features extracted from the Part-of-Speech (POS) tagger and gazetteers. Context patterns generated from the unlabeled data using an active learning method are also used as the features in each of the classifiers. A semi-supervised method is proposed to describe the measures to automatically select effective unlabeled documents as well as sentences from the unlabeled data. Finally, the supervised models are combined together into a final system by defining appropriate weighted voting technique. Experimental results for a resource-poor language like Bengali show the effectiveness of the proposed approach with the overall recall, precision and F-measure values of 93.81%, 92.18% and 92.98%, respectively.Keywords
This publication has 4 references indexed in Scilit:
- A web-based Bengali news corpus for named entity recognitionLanguage Resources and Evaluation, 2008
- Shallow parsing with conditional random fields2003
- Named entity recognition through classifier combination2003
- The Nature of Statistical Learning Theory1995