Top2Vec: Distributed Representations of Topics
Author:
Preprint
Abstract
Topic modeling is used for discovering latent semantic structure, usually referred to as topics, in a large collection of documents. The most widely used methods are Latent Dirichlet Allocation and Probabilistic Latent Semantic Analysis. Despite their popularity they have several weaknesses. In order to achieve optimal results they often require the number of topics to be known, custom stop-word lists, stemming, and lemmatization. Additionally these methods rely on bag-of-words representation of documents which ignore the ordering and semantics of words. Distributed representations of documents and words have gained popularity due to their ability to capture semantics of words and documents. We present $\texttt{top2vec}$, which leverages joint document and word semantic embedding to find $\textit{topic vectors}$. This model does not require stop-word lists, stemming or lemmatization, and it automatically finds the number of topics. The resulting topic vectors are jointly embedded with the document and word vectors with distance between them representing semantic similarity. Our experiments demonstrate that $\texttt{top2vec}$ finds topics which are significantly more informative and representative of the corpus trained on than probabilistic generative models.
Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach2019
- UMAP: Uniform Manifold Approximation and ProjectionJournal of Open Source Software, 2018
- hdbscan: Hierarchical density based clusteringJournal of Open Source Software, 2017
- Improving Distributional Similarity with Lessons Learned from Word EmbeddingsTransactions of the Association for Computational Linguistics, 2015
- An information-theoretic perspective of tf–idf measuresInformation Processing & Management, 2003