Augmentation of Topic Modeling: Comparing The Traditional and Their Word Embedding Oriented Approaches

Abstract

This paper compares the efficacy of classical topic modeling approaches namely the Latent Dirichlet Allocation (LDA) and Non-Negative Matrix Factorization (NMF) with their respective word embedding enhanced techniques. The LDA and NMF are based on word frequency and co-occurrence patterns, but ignore the semantic similarity between words. To address these limitations, the proposed methods enhance the LDA and NMF with Word2Vec embeddings by capturing the semantic relationships, and overseeing the improvement in topic coherence. This study therefore examines the comparable strengths of these approaches and shows how word embeddings can improve overall topic modeling. Our experimental results have shown that the Word2Vec enhanced LDA and NMF improve the coherence scores over their traditional counterparts.