An evaluation of Hadoop cluster efficiency in document clustering using parallel K-means

TLDR: This study design and experiment a parallel k-means algorithm using MapReduce programming model and compared the result with sequential k-Means for clustering varying size of document dataset and demonstrates that proposed k- means obtains higher performance and outperformed sequential k -means while clustering documents.

Abstract: One of the significant data mining techniques is clustering. Due to digitalization and globalization of each work space, large datasets are being generated rapidly. Such large dataset clustering is a challenge for traditional sequential clustering algorithms as it requires large execution time to cluster such datasets. Distributed parallel architectures and algorithms are thus helpful to achieve performance and scalability requirement of clustering large datasets. In this study, we design and experiment a parallel k-means algorithm using MapReduce programming model and compared the result with sequential k-means for clustering varying size of document dataset. The result demonstrates that proposed k-means obtains higher performance and outperformed sequential k-means while clustering documents.