Matrix Factorization Techniques for Recommender Systems

With the growth of online social network platforms and applications, large amounts of textual user-generated content are created daily in the form of comments, reviews, and short-text messages. As a result, users often find it challenging to discover useful information or more on the topic being discussed from such content. Machine learning and natural language processing algorithms are used to analyze the massive amount of textual social media data available online, including topic modeling techniques that have gained popularity in recent years. This paper investigates the topic modeling subject and its common application areas, methods, and tools. Also, we examine and compare five frequently used topic modeling methods, as applied to short textual social data, to show their benefits practically in detecting important topics. These methods are latent semantic analysis, latent Dirichlet allocation, non-negative matrix factorization, random projection, and principal component analysis. Two textual datasets were selected to evaluate the performance of included topic modeling methods based on the topic quality and some standard statistical evaluation metrics, like recall, precision, F-score, and topic coherence. As a result, latent Dirichlet allocation and non-negative matrix factorization methods delivered more meaningful extracted topics and obtained good results. The paper sheds light on some common topic modeling methods in a short-text context and provides direction for researchers who seek to apply these methods.

/pdf/using-topic-modeling-methods-for-short-text-data-a-15j608gync.pdf

Using Topic Modeling Methods for Short-Text Data: A Comparative Analysis.

One of the most crucial problem on training personalized response generation models for conversational robots is the lack of large scale personal conversation data. To address the problem, we propose a two-phase approach, namely initialization then adaptation, to first pre-train an optimized RNN encoder-decoder model (LTS model) in a large scale conversational data for general response generation and then fine-tune the model in a small scale personal conversation data to generate personalized responses. For evaluation, we propose a novel human aided method, which can be seen as a quasi-Turing test, to evaluate the performance of the personalized response generation models. Experimental results show that the proposed personalized response generation model outperforms the state-of-the-art approaches to language model personalization and persona-based neural conversation generation on the automatic evaluation, offline human judgment and the quasi-Turing test.

Neural personalized response generation as domain adaptation

In this era, the proliferating role of social media in our lives has popularized the posting of the short text. The short texts contain limited context with unique characteristics which makes them difficult to handle. Every day billions of short texts are produced in the form of tags, keywords, tweets, phone messages, messenger conversations social network posts, etc. The analysis of these short texts is imperative in the field of text mining and content analysis. The extraction of precise topics from large-scale short text documents is a critical and challenging task. The conventional approaches fail to obtain word co-occurrence patterns in topics due to the sparsity problem in short texts, such as text over the web, social media like Twitter, and news headlines. Therefore, in this paper, the sparsity problem is ameliorated by presenting a novel fuzzy topic modeling (FTM) approach for short text through fuzzy perspective. In this research, the local and global term frequencies are computed through a bag-of-words (BOW) model. To remove the negative impact of high dimensionality on the global term weighting, the principal component analysis is adopted; thereafter the fuzzy c-means algorithm is employed to retrieve the semantically relevant topics from the documents. The experiments are conducted over the three real-world short text datasets: the snippets dataset is in the category of small dataset whereas the other two datasets, Twitter and questions, are the bigger datasets. Experimental results show that the proposed approach discovered the topics more precisely and performed better as compared to other state-of-the-art baseline topic models such as GLTM, CSTM, LTM, LDA, Mix-gram, BTM, SATM, and DREx+LDA. The performance of FTM is also demonstrated in classification, clustering, topic coherence and execution time. FTM classification accuracy is 0.95, 0.94, 0.91, 0.89 and 0.87 on snippets dataset with 50, 75, 100, 125 and 200 number of topics. The classification accuracy of FTM on questions dataset is 0.73, 0.74, 0.70, 0.68 and 0.78 with 50, 75, 100, 125 and 200 number of topics. The classification accuracies of FTM on snippets and questions datasets are higher than state-of-the-art baseline topic models.

Fuzzy topic modeling approach for text mining over short text

Artificial intelligence (AI) techniques have grown rapidly in recent years in the context of computing with smart mobile phones that typically allows the devices to function in an intelligent manner. Popular AI techniques include machine learning and deep learning methods, natural language processing, as well as knowledge representation and expert systems, can be used to make the target mobile applications intelligent and more effective. In this paper, we present a comprehensive view on “mobile data science and intelligent apps” in terms of concepts and AI-based modeling that can be used to design and develop intelligent mobile applications for the betterment of human life in their diverse day-to-day situation. This study also includes the concepts and insights of various AI-powered intelligent apps in several application domains, ranging from personalized recommendation to healthcare services, including COVID-19 pandemic management in recent days. Finally, we highlight several research issues and future directions relevant to our analysis in the area of mobile data science and intelligent apps. Overall, this paper aims to serve as a reference point and guidelines for the mobile application developers as well as the researchers in this domain, particularly from the technical point of view.

/pdf/mobile-data-science-and-intelligent-apps-concepts-ai-based-19q4t82efp.pdf

Mobile Data Science and Intelligent Apps: Concepts, AI-Based Modeling and Research Directions

With the development of science and technology, the popularity of smart phones has made exponential growth in mobile phone application market. How to help users to select applications they prefer has become a hot topic in recommendation algorithm. As traditional recommendation algorithms are based on popularity and download, they inadvertently fail to recommend the desirable applications. At the same time, many users tend to pay more attention to permissions of those applications, because of some privacy and security reasons. There are few recommendation algorithms which take account of apps’ permissions, functionalities and users’ interests altogether. Some of them only consider permissions while neglecting the users’ interests, others just perform linear combination of apps’ permissions, functionalities and users’ interests to implement top-N recommendation. In this paper, we devise a recommendation method based on both permissions and functionalities. After demonstrating the correlation of apps’ permissions and users’ interests, we design an app risk score calculating method ARSM based on app-permission bipartite graph model. Furthermore, we propose a novel matrix factorization algorithm MFPF based on users’ interests, apps’ permissions and functionalities to handle personalized app recommendation. We compare our work with some of the state-of-the-art recommendation algorithms, and the results indicate that our work can improve the recommendation accuracy remarkably.

Personalized app recommendation based on app permissions

Short texts have become the prevalent format of information on the Internet. Inferring the topics of this type of messages becomes a critical and challenging task for many applications. Due to the length of short texts, conventional topic models (e.g., latent Dirichlet allocation and its variants) suffer from the severe data sparsity problem which makes topic modeling of short texts difficult and unreliable. Recently, word embeddings have been proved effective to capture semantic and syntactic information about words, which can be used to induce similarity measures and semantic correlations among words. Enlightened by this, in this paper, we design a novel model for short text topic modeling, referred as Conditional Random Field regularized Topic Model (CRFTM). CRFTM not only develops a generalized solution to alleviate the sparsity problem by aggregating short texts into pseudo-documents, but also leverages a Conditional Random Field regularized model that encourages semantically related words to share the same topic assignment. Experimental results on two real-world datasets show that our method can extract more coherent topics, and significantly outperform state-of-the-art baselines on several evaluation metrics.

Incorporating word embeddings into topic modeling of short text

This article presents an unsupervised multi-view hierarchical embedding (UMHE) framework to sufficiently reveal the intrinsic topical knowledge in social events. Event-oriented topics are highly related to such events as it can provide explicit descriptions of what have happened in social community. In many real-world cases, however, it is difficult to include all attributes of microblogs, more often, textual aspects only are available. Traditional topic modelling methods have failed to generate event-oriented topics with the textual aspects, since the inherent relations between topics are often overlooked in these methods. Meanwhile, the metrics in original word vocabulary space might not effectively capture semantic distances. Our UMHE framework overcomes the severe information deficiency and poor feature representation. The UMHE first develops a multi-view Bayesian rose tree to preliminarily generate prior knowledge for latent topics and their relations. With such prior knowledge, we design an unsupervised translation-based hierarchical embedding method to make a better representation of these latent topics. By applying self-adaptive spectral clustering on the embedding space and the original space concomitantly, we eventually extract event-oriented topics in word distributions to express social events. Our framework is purely data-driven and unsupervised, without any external knowledge. Experimental results on TREC Tweets2011 dataset and Sina Weibo dataset demonstrate that the UMHE framework can construct hierarchical structure with high fitness, but also yield topic embeddings with salient semantics; therefore, it can derive event-oriented topics with meaningful descriptions.

/pdf/mining-event-oriented-topics-in-microblog-stream-with-2k6xrc1850.pdf

Mining Event-Oriented Topics in Microblog Stream with Unsupervised Multi-View Hierarchical Embedding

Topic models with sparsity enhancement have been proven to be effective at learn- ing discriminative and coherent latent top- ics of short texts, which is critical to many scientific and engineering applica- tions. However, the extensions of these models require carefully tailored graphi- cal models and re-deduced inference al- gorithms, limiting their variations and ap- plications. We propose a novel sparsity- enhanced topic model, Neural Sparse Top- ical Coding (NSTC) base on a sparsity- enhanced topic model called Sparse Top- ical Coding (STC). It focuses on replac- ing the complex inference process with the back propagation, which makes the model easy to explore extensions. Moreover, the external semantic information of words in word embeddings is incorporated to im- prove the representation of short texts. To illustrate the flexibility offered by the neu- ral network based framework, we present three extensions base on NSTC without re-deduced inference algorithms. Experi- ments on Web Snippet and 20Newsgroups datasets demonstrate that our models out- perform existing methods.

/pdf/neural-sparse-topical-coding-35rn2mzoya.pdf

Neural Sparse Topical Coding

Sparse topic models (STMs) are widely used for learning a semantically rich latent sparse representation of short texts in large scale, mainly by imposing sparse priors or appropriate regularizers on topic models. However, it is difficult for these STMs to model the sparse structure and pattern of the corpora accurately, since their sparse priors always fail to achieve real sparseness, and their regularizers bypass the prior information of the relevance between sparse coefficients. In this paper, we propose a novel Bayesian hierarchical topic models called Bayesian Sparse Topical Coding with Poisson Distribution (BSTC-P) on the basis of Sparse Topical Coding with Sparse Groups (STCSG). Different from traditional STMs, it focuses on imposing hierarchical sparse prior to leverage the prior information of relevance between sparse coefficients. Furthermore, we propose a sparsity-enhanced BSTC, Bayesian Sparse Topical Coding with Normal Distribution (BSTC-N), via mathematic approximation. We adopt superior hierarchical sparse inducing prior, with the purpose of achieving the sparsest optimal solution. Experimental results on datasets of Newsgroups and Twitter show that both BSTC-P and BSTC-N have better performance on finding clear latent semantic representations. Therefore, they yield better performance than existing works on document classification tasks.

/pdf/bayesian-sparse-topical-coding-6vle07b9eo.pdf

Gang Tian

Papers

Personalized app recommendation based on app permissions

Incorporating word embeddings into topic modeling of short text

Mining Event-Oriented Topics in Microblog Stream with Unsupervised Multi-View Hierarchical Embedding

Neural Sparse Topical Coding

Bayesian Sparse Topical Coding