A novel hybrid clustering algorithm for microblog topic detection
- 1 January 2017
- conference paper
- Published by AIP Publishing in AIP Conference Proceedings
- Vol. 1890, 040074
Abstract
Microblog has the characteristics of large scale, various topics and too much topic-unrelated texts included. So we propose a three -layer hybrid clustering algorithm to replace the original ones used in the topic detection models which can hardly handle microblog. We apply the K-means algorithm in clustering the microblog texts by their topics in the first layer. And in the second layer, we use the agglomerative nesting algorithm to merge the small clusters consisting of texts of the same topic. The first two layers also remove most noise, reducing their further impact on the K-means in the third layer, which reassigns the texts assigned to the wrong cluster. Experiments show our algorithm outperforms some related traditional algorithms on the clustering of real dataset and functions perfectly in the topic detection.Keywords
This publication has 6 references indexed in Scilit:
- Micro-blog topic detection method based on BTM topic model and K-means clustering algorithmAutomatic Control and Computer Sciences, 2016
- Combination of singular value decomposition and K-means clustering methods for topic detection on Twitter2015
- Chinese Microblog Topic Detection Based on the Latent Semantic Analysis and Structural PropertyJournal of Networks, 2013
- Taking Topic Detection From Evaluation to Practice2005
- A new cluster validity index for the fuzzy c-meanPattern Recognition Letters, 1998
- Algorithm AS 136: A K-Means Clustering AlgorithmJournal of the Royal Statistical Society Series C: Applied Statistics, 1979