A study of word embedding models for measuring topic coherence
Open Access
- 19 May 2026
- journal article
- research article
- Published by Springer Nature in Knowledge and Information Systems
- Vol. 68 (1) , 160
Abstract
Topic modeling has emerged as a crucial tool in the field of natural language processing, enabling the automatic discovery of latent structures in large textual corpora. However, determining the quality of the topics remains a significant challenge, particularly in measuring the coherence of the top words of the extracted topics. Early efforts relied on human judgments, but these approaches are resource-intensive. Automated coherence metrics have since been developed. For example, some measures exploit word co-occurrence, while other methods are grounded in distributional semantics (e.g., employing word embeddings). In this study, we thoroughly explore the application of embedded representations to evaluate the quality of topics. While a number of isolated studies have analyzed the role of specific word representation techniques for measuring topic coherence, a complete picture of their effectiveness is still lacking. This work brings together different embedding-based approaches, including Word2Vec, FastText, GloVe, and BERT, which had been studied separately, and extends prior research by incorporating additional models, such as RoBERTa, ALBERT and MPNET. Topic coherence is measured by computing similarity scores between word embeddings, thus obtaining rich semantic associations that traditional measures may overlook. Our analysis demonstrates that these methods are as effective as, and often surpass, classical coherence measures. Our results contribute to a growing body of research advocating for advanced semantic representations as robust alternatives to traditional approaches in evaluating topic model coherence.Keywords
Funding Information
- Agencia Estatal de Investigación (PID2022-137061OB-C22)
- Consellería de Cultura, Educación e Ordenación Universitaria, Xunta de Galicia (ED431G 2023/04, ED431C 2022/19)
- Universidade de Santiago de Compostela
This publication has 24 references indexed in Scilit:
- Self-supervised learning for neural topic models with variance–invariance–covariance regularizationKnowledge and Information Systems, 2025
- Sparse dynamic topic model with topic birth and death over timeKnowledge and Information Systems, 2025
- Mining technology trends in scientific publications: a graph propagated neural topic modeling approachKnowledge and Information Systems, 2024
- A word embedding-based approach to cross-lingual topic modelingKnowledge and Information Systems, 2021
- Incorporating word embeddings into topic modeling of short textKnowledge and Information Systems, 2018
- Exploring the Space of Topic Coherence Measures2015
- Probabilistic topic modelsCommunications of the ACM, 2012
- Topic Significance Ranking of LDA Generative Models2009
- Indexing by latent semantic analysisJournal of the American Society for Information Science, 1990
- Learning representations by back-propagating errorsNature, 1986