A parallel Probabilistic Latent Semantic Analysis method on MapReduce platform

TLDR: The empirical experiment results show that when the training dataset is large, learning the probability distributions of PLSA model in a parallel way can achieve almost linear speedups and thus provides a practical solution to large-scale data analysis applications.

Abstract: Probabilistic Latent Semantic Analysis (PLSA) is a powerful statistical technique to analyze relation between co-occurrence data, and has wide usages in automated information processing tasks. However it involves non-trivial computation and is often difficult and time-consuming to train when the dataset is big. MapReduce is a computing framework designed by Google which aims to provide a distributed solution to practically large-scale data analysis tasks using clusters of computers. In this work, we addressed the scalability problem of PLSA by proposing and implementing a parallel method to train PLSA under the MapReduce computing framework. The empirical experiment results show that when the training dataset is large, learning the probability distributions of PLSA model in a parallel way can achieve almost linear speedups and thus provides a practical solution to large-scale data analysis applications.