The Effects of Preprocessing on Turkish and English News Data
Author:
- 30 April 2023
- journal article
- research article
- Published by Sakarya University Journal of Computer and Information Sciences in Sakarya University Journal of Computer and Information Sciences
- Vol. 6 (1) , 59-66
Abstract
In a standard text classification (TC) study, preprocessing is one of the key components to improve performance. This study aims to look at how preprocessing effects TC according to news text, text language, and feature selection. All potential combinations of commonly used preprocessing techniques are compared on one domain, namely news data, and in two different news datasets for this aim. Preprocessing technique contributions to classification performance at multiple feature sizes, possible interconnections among these techniques, and technique dependency on corresponding languages are all evaluated in this way. Using best combinations of preprocessing techniques rather than using or not using them all, experimental studies on public datasets reveals that, choosing best combinations of preprocessing techniques can improve classification accuracy significantly.Keywords
This publication has 19 references indexed in Scilit:
- Selection of the most relevant terms based on a max-min ratio metric for text classificationExpert Systems with Applications, 2018
- Feature selection based on a normalized difference measure for text classificationInformation Processing & Management, 2017
- A novel text mining approach based on TF-IDF and Support Vector Machine for news classification2016
- TTC-3600: A new benchmark dataset for Turkish text categorizationJournal of Information Science, 2015
- The impact of preprocessing on text classificationInformation Processing & Management, 2014
- Analysis of preprocessing methods on classification of Turkish texts2011
- Information gain and divergence-based feature selection for machine learning-based text categorizationInformation Processing & Management, 2006
- Text categorization with Support Vector Machines: Learning with many relevant features1998
- An algorithm for suffix strippingProgram: electronic library and information systems, 1980
- A vector space model for automatic indexingCommunications of the ACM, 1975