Tuning Language Representation Models for Classification of Turkish News
Authors:
Open Access
- 19 February 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 402-407
Abstract
Pre-trained language representation models are very efficient in learning language representation independent from natural language processing tasks to be performed. The language representation models such as BERT and DistilBERT have achieved amazing results in many language understanding tasks. Studies on text classification problems in the literature are generally carried out for the English language. This study aims to classify the news in the Turkish language using pre-trained language representation models. In this study, we utilize BERT and DistilBERT by tuning both models for the text classification task to learn the categories of Turkish news with different tokenization methods. We provide a quantitative analysis of the performance of BERT and DistilBERT on the Turkish news dataset by comparing the models in terms of their representation capability in the text classification task. The highest performance is obtained with DistilBERT with an accuracy of 97.4%.Keywords
This publication has 13 references indexed in Scilit:
- Deep learning for sentiment analysis: A surveyWIREs Data Mining and Knowledge Discovery, 2018
- Spam Detection on Social Media Using Semantic Convolutional Neural NetworkInternational Journal of Knowledge Discovery in Bioinformatics, 2018
- Deep Learning for Extreme Multi-label Text Classification2017
- Twitter spam detection based on deep learning2017
- TTC-3600: A new benchmark dataset for Turkish text categorizationJournal of Information Science, 2015
- Text Classification Using Machine Learning Methods-A Survey2014
- Glove: Global Vectors for Word Representation2014
- Novel comment filtering approach based on outlier on streaming data2012
- Learning-based pronoun resolution for Turkish with a comparative evaluationComputer Speech & Language, 2009
- Context-sensitive learning methods for text categorizationACM Transactions on Information Systems, 1999