Alper Kursat Uysal

Annotation agreement is an important indicator of annotation quality in supervised text classification; however, its effect on financial sentiment classification has received limited attention. This study investigates the influence of annotation agreement on classification performance using the Financial PhraseBank dataset. Three mutually exclusive datasets representing high, medium, and low agreement annotations were constructed to eliminate duplicated instances across agreement levels. Nine within-level and cross-level experimental settings were evaluated using two preprocessing configurations, TF-IDF unigram features, and eight traditional machine learning classifiers. Experimental results showed that classification performance consistently decreased as annotation agreement declined. Models evaluated on higher-agreement test datasets consistently achieved better performance than those evaluated on lower-agreement datasets, regardless of the annotation agreement level of the training data. Linear SVM and SGD (Hinge) achieved the highest Macro-F1 scores across most experimental settings, whereas lemmatization provided only limited and inconsistent improvements. These findings demonstrate that annotation agreement is an important characteristic of annotated datasets and should be explicitly considered when constructing benchmark datasets, evaluating financial sentiment classification models, and interpreting experimental results.

Hotel stars are key indicators for assessing locations, facilities, and other attributes. Although the World Tourism Organization has long advocated for such systems, no consensus has been reached on standardizing them. Italy and Spain use their own systems. Kazakhstan has no system. Since visitors rate hotels, Web 2.0 implementations have started using their rating systems. Hotels in countries without a standard system would be negatively affected and would be vulnerable to manipulation by Web 2.0. Topic modeling and sentiment analysis are implemented to explore how hotel stars converge or diverge regarding hotel attributes mentioned by visitors. TripAdvisor’s 5,894 online reviews of Astana’s hotels in Kazakhstan between 2006 and 2023 are used. The study pinpoints the same attributes as remaining issues, even though hotels differ in stars. Customers’ sentiments about derived hotel attributes mostly converge for hotels rated 2, 3, and 4 stars; however, they diverge in 5-star hotels. Specifically, even though hotels with 2, 3, or 4 stars differ, they share similar issues. Thus, official rating systems should be adopted in Kazakhstan soon. Kazakhstan’s hotels can directly adopt an official rating system to enhance attribute qualities.

Sentiment classification has become one of the most widely studied areas in text classification, especially in recent years. This study presents extensive experiments in sentiment analysis, investigating the performance of seven state-of-the-art sentiment analyzers (TextBlob, VADER, AFINN, Stanza, Nlptown, Sentistrength, and Flair) in Stage 1, and an ensemble approach in Stage 2, using multilingual restaurant reviews from Almaty, Kazakhstan. The reviews, either originally written in English or translated from Russian, are analyzed across various sections, including HEAD, TEXT, and their combinations (HEAD+TEXT). The results of Stage 2 ensemble methods demonstrate clear advantages of carefully selected ensembles over individual sentiment analyzers. Specifically, the highest Micro-F1 score for English reviews was 0.733 in the TEXT section, achieved by the ensemble TextBlob+Stanza+Nlptown+Sentistrength. The highest Macro-F1 score for English reviews was 0.684, achieved by the same ensemble in the TEXT section. For Russian reviews, the highest Micro-F1 score was 0.703 in the HEAD+TEXT combination, and the highest Macro-F1 score was 0.642 in the TEXT section, both achieved by the ensemble TextBlob+Stanza+Nlptown+Sentistrength. These findings highlight that the performance of sentiment analyzers varies depending on the original language and the corresponding review section.

Attractions in the tourism industry are one of the components that motivate tourists to visit destinations, such as entertainment, natural, cultural, and historical richness. For such reasons, people decide to visit unique destinations and spend time there. Almaty, the largest city of Kazakhstan, is one of the significant attraction centers of the Central Asia region, offering tourists unique and pleasant features with several tourist attractions. This study aims to analyze online user reviews of tourist attractions in Almaty, Kazakhstan, using machine learning and text mining methods. The primary focus is on identifying the main thematic clusters of reviews and their sentiment and comparing these themes with the types of attractions: historical, natural, and man-made. A total of 7,515 reviews were collected from the TripAdvisor website. The data was processed using sentiment analysis, topic modeling, and hierarchical clustering methods. The analysis revealed that 38% of the reviews were related to natural attractions, 34% to man-made, and 28% to historical ones. The most positive reviews were associated with natural attractions, while historical and man-made attractions received 79.38% and 81.40% positive reviews, respectively. In addition, the items that make up these attractions are identified, and their sentiment levels are pointed out. In addition to this situation, visitors have the most positive expressions for natural attractions, especially landscapes and lakes. The findings emphasize the importance of considering review themes to improve the quality of tourist services and to enhance the positive image of Almaty as a tourist destination.

The primary objective of employing multiple classifier systems (MCS) in pattern recognition is to enhance classification accuracy. Dynamic classifier selection (DCS) and dynamic ensemble selection (DES) are two purposeful forms of multiple classifier systems. While DES involves the selection of a classifier set followed by decision combination, DCS opts for the choice of a single competent classifier, eliminating the necessity for classifier combination. As a consequence, DCS methods exhibit superior efficiency in terms of processing time and memory usage compared to DES methods. Moreover, a substantial performance gap exists between the performance of Oracle and both DES and DCS methods. In this study, we introduce a novel method termed dynamic classifier selection technique-decision quotient (DCS-DQ) for text classification based on dynamic classifier selection. Our experimental investigation encompasses four distinct text datasets, with classification accuracy and macro F1-score serving as the primary evaluation criteria. The proposed DCS-DQ method is subjected to comparison with seven state-of-the-art DCS methods. Based on our empirical findings, the DCS-DQ method outperforms the other seven DCS methods in terms of classification accuracy across the majority of feature sizes. Notably, in the Reuters dataset, the classification accuracy of DCS-DQ surpasses that of other DCS methods for all feature sizes except when the feature size is 100. Similarly, in the Ohsumed dataset, the DCS-DQ method demonstrates significant performance improvement, with an accuracy value of 77.02% for 3000 features compared to the maximum accuracy value of 72.74% achieved by the DCS method MCB. Additionally, the performance of the proposed DCS-DQ method closely aligns with the oracle performance compared to the other methods. In conclusion, our proposed DCS-DQ method holds promise for significantly improving classification accuracy in text classification literature.

Metin verilerinin sınıflar arasında dağılımı genellikle eşit değildir. Bu durum, metin sınıflandırma işleminde sınıflandırıcıların performansına olumsuz yansımaktadır. Dengesiz metin sınıflandırma ile ilgili birçok çalışma yapılmıştır. Metin sınıflandırma işleminin önemli aşamalarından olan öznitelik seçim aşaması, dengesiz metin probleminde de kritik öneme sahiptir. Öznitelik seçme metotlarının dengesiz metinlerin sınıflandırılması üzerindeki etkisi bu çalışmada etraflıca araştırılmıştır. Bu doğrultuda, iki farklı veri seti üzerinde üç farklı sınıflandırıcı ve dokuz farklı öznitelik seçim metodu ile birçok deney yapılmıştır. Ayrıca öznitelik seçim yöntemlerinin başarıları farklı öznitelik sayılarında da gözlemlenmiştir. NDM, DFSS, PFS, POISSON, CHI2, IG, GINI, DFS ve MDFS olarak adlandırılan 9 farklı öznitelik seçim metodu değerlendirilmiştir. Destek Vektör Makinesi (SVM), Karar Ağacı (DTREE) ve Basit Bayes (MNB) sınıflandırıcıları ile deneysel sonuçlar elde edilmiştir. Reuters-21578 veri setinde DFS ve CHI2 öznitelik seçim yöntemleri Makro-F1 değerlendirme metriği üzerinden yaklaşık en yüksek 80 değerini alırken, SPAM SMS veri setinde, DFS öznitelik seçim yöntemi en yüksek skor olarak 95 ve CHI2 öznitelik seçim yöntemi 94 değerlerini almıştır. Öznitelik seçme metotlarından DFS ve CHI2’nin dengesiz metin sınıflandırmada daha başarılı olduğu görülmektedir.

This study aims to place EFL learners along an affective continuum via machine learning methods and present a new dataset about affective characteristics of EFL learners. In line with the purposes, written self-reports of 475 students from 5 different faculties in 3 universities in Turkey were collected and manually assigned by the researchers to one of the labels (positive, negative, or neutral). As a result, two combinations of the same dataset (AC-2 and AC-3) including different numbers of classes were used for the assessment of automatic classification approaches. Results revealed that automatic classification confirmed the manual classification to a great extent and machine learning methods could be used to classify EFL students along an affective continuum according to their affective characteristics. Maximum accuracy rate of automatic classification is 90.06% on AC-2 dataset including two classes. Similarly, on AC-3 dataset including three classes, maximum accuracy rate of classification is 71.79%. Last, the top-10 features/words obtained by feature selection methods are highly discriminative in terms of assessing student feelings for EFL learning. It could be stated that there is not an existing study in which feature selection methods and classifiers are used in the literature to automatically classify EFL learners’ feelings.

In recent years, short texts are everywhere, especially in social media networks. Short text classification is an essential task for various applications related to the operations on short text documents. In many cases, using the entire feature set causes the high dimensionality problem in short text data. This problem reason of time‐consuming and negatively impacts the performance of classifiers. This study presents an effective feature selection algorithm called XY method, which represents the features on XY line and calculates the distance of a feature to the XY line. Also, a value named λ is calculated. According to this value, the terms are divided into different regions such as negative, positive, and third to determine their discrimination capability. The novel XY method aims to select as few terms as possible in the negative region. The proposed method is evaluated using four different short text datasets with Macro‐F1 success measure. In comparisons with other existing feature selection algorithms such as chi‐square, information gain, deviation from Poisson distribution, recently proposed max‐min ratio, and distinguishing feature selector demonstrate that the XY method achieves either better or competitive performance in significantly reduced various feature sizes.

As the huge dimensionality of textual data restrains the classification accuracy, it is essential to apply feature selection (FS) methods as dimension reduction step in text classification (TC) domain. Most of the FS methods for TC contain several number of probabilities. In this study, we proposed a new FS method named as Extensive Feature Selector (EFS), which benefits from corpus-based and class-based probabilities in its calculations. The performance of EFS is compared with nine well-known FS methods, namely, Chi-Squared (CHI2), Class Discriminating Measure (CDM), Discriminative Power Measure (DPM), Odds Ratio (OR), Distinguishing Feature Selector (DFS), Comprehensively Measure Feature Selection (CMFS), Discriminative Feature Selection (DFSS), Normalised Difference Measure (NDM) and Max–Min Ratio (MMR) using Multinomial Naive Bayes (MNB), Support-Vector Machines (SVMs) and k-Nearest Neighbour (KNN) classifiers on four benchmark data sets. These data sets are Reuters-21578, 20-Newsgroup, Mini 20-Newsgroup and Polarity. The experiments were carried out for six different feature sizes which are 10, 30, 50, 100, 300 and 500. Experimental results show that the performance of EFS method is more successful than the other nine methods in most cases according to micro-F1 and macro-F1 scores.