Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Data Mining - Concepts and Techniques.

Matrix Factorization Techniques for Recommender Systems

This chapter introduces the subject of statistical pattern recognition (SPR). It starts by considering how features are defined and emphasizes that the nearest neighbor algorithm achieves error rates comparable with those of an ideal Bayes’ classifier. The concepts of an optimal number of features, representativeness of the training data, and the need to avoid overfitting to the training data are stressed. The chapter shows that methods such as the support vector machine and artificial neural networks are subject to these same training limitations, although each has its advantages. For neural networks, the multilayer perceptron architecture and back-propagation algorithm are described. The chapter distinguishes between supervised and unsupervised learning, demonstrating the advantages of the latter and showing how methods such as clustering and principal components analysis fit into the SPR framework. The chapter also defines the receiver operating characteristic, which allows an optimum balance between false positives and false negatives to be achieved.

Statistical Pattern Recognition

With the growing volume of online information, recommender systems have been an effective strategy to overcome information overload. The utility of recommender systems cannot be overstated, given their widespread adoption in many web applications, along with their potential impact to ameliorate many problems related to over-choice. In recent years, deep learning has garnered considerable interest in many research fields such as computer vision and natural language processing, owing not only to stellar performance but also to the attractive property of learning feature representations from scratch. The influence of deep learning is also pervasive, recently demonstrating its effectiveness when applied to information retrieval and recommender systems research. The field of deep learning in recommender system is flourishing. This article aims to provide a comprehensive review of recent research efforts on deep learning-based recommender systems. More concretely, we provide and devise a taxonomy of deep learning-based recommendation models, along with a comprehensive summary of the state of the art. Finally, we expand on current trends and provide new perspectives pertaining to this new and exciting development of the field.

Deep Learning Based Recommender System: A Survey and New Perspectives

Implicit feedbacks have recently received much attention in recommendation communities due to their close relationship with real industry problem settings. However, most works only exploit users’ homogeneous implicit feedbacks such as users’ transaction records from “bought” activities, and ignore the other type of implicit feedbacks like examination records from “browsed” activities. The latter are usually more abundant though they are associated with high uncertainty w.r.t. users’ true preferences. In this paper, we study a new recommendation problem called heterogeneous implicit feedbacks (HIF), where the fundamental challenge is the uncertainty of the examination records. As a response, we design a novel preference learning algorithm to learn a confidence for each uncertain examination record with the help of transaction records. Specifically, we generalize Bayesian personalized ranking (BPR), a seminal pairwise learning algorithm for homogeneous implicit feedbacks, and learn the confidence adaptively, which is thus called adaptive Bayesian personalized ranking (ABPR). ABPR has the merits of uncertainty reduction on examination records and accurate pairwise preference learning on implicit feedbacks. Experimental results on two public data sets show that ABPR is able to leverage uncertain examination records effectively, and can achieve better recommendation performance than the state-of-the-art algorithm on various ranking-oriented evaluation metrics.

Adaptive Bayesian personalized ranking for heterogeneous implicit feedbacks

Sensor deployment is a fundamental issue in sensor networks. Studies on the deployment problem are concentrated deploying sensors to cover assigned areas or set of points efficiently. In this paper, we discuss the sensor deployment problem in the context of target tracking. Firstly, we propose a sensor deployment optimization strategy based on target involved virtual force algorithm (TIVFA). The TIVFA algorithm can dynamically adjust sensor networks configuration according to terrains, intelligence and those detected maneuvering targets in order to improve coverage and detection probability. Secondly, we present an improved sensor-ranking algorithm as well as a sensor protecting strategy with targets importance sequence in consideration. The TIVFA algorithm, together with the sensor protecting strategy, can produce an efficient and robust sensor network. Experiment results sufficiently demonstrate the effectiveness of the proposed approach.

https://isif.org/fusion/proceedings/fusion05CD/Content/papers/f42d27627f8f44310f2cad105095.pdf

Sensor deployment optimization for detecting maneuvering targets

Fisher linear discriminant analysis (FDA) and its kernel extension--kernel discriminant analysis (KDA)--are well known methods that consider dimensionality reduction and classification jointly. While widely deployed in practical problems, there are still unresolved issues surrounding their efficient implementation and their relationship with least mean squares procedures. In this paper we address these issues within the framework of regularized estimation. Our approach leads to a flexible and efficient implementation of FDA as well as KDA. We also uncover a general relationship between regularized discriminant analysis and ridge regression. This relationship yields variations on conventional FDA based on the pseudoinverse and a direct equivalence to an ordinary least squares estimator.

/pdf/regularized-discriminant-analysis-ridge-regression-and-oifqwowvqi.pdf

Regularized Discriminant Analysis, Ridge Regression and Beyond

The problem of uncertainty knowledge has been tackled for a long time by philosophers, logicians and mathematicians. Recently it becomes a crucial issue for computer scientists, particularly in the area of artificial intelligence (AI). The set pair analysis (SPA) theory, proposed by Keqin Zhao, is a novel uncertainty theory. The core of this theory is to consider certainties and uncertainties as a certain-uncertain system, and to depict uniformly all kinds of uncertainties such as random uncertainty, fuzzy uncertainty, indeterminate-known uncertainty, unknown and unexpected incident uncertainty, and uncertainty that results from imperfective information, using a connection degree formula that can fully embody its idea. SPA has been applied to many fields successfully such as industry, agriculture, forestry, education, physical education, military affairs, traffic, data fusion, decision-making, forecasting, comprehensive evaluation, and network planning, etc. The reason is that there exists abundant systems information such as system structure information, system theory information, etc, in SPA. In this paper, the systems information in SPA is discussed, and its applications are also given.

Systems information in set pair analysis and its applications

Collaborative recommendation has attracted various research works in recent years However, an important problem setting, ie, "a user examined several items but only rated a few", has not received much attention yet We coin this problem heterogeneous collaborative recommendation (HCR) from the perspective of users' heterogeneous feedbacks of implicit examinations and explicit ratings In order to fully exploit such different types of feedbacks, we propose a novel and generic solution called compressed knowledge transfer via factorization machine (CKT-FM) Specifically, we assume that the compressed knowledge of user homophily and item correlation, ie, user groups and item sets behind two types of feedbacks, are similar and then design a two-step transfer learning solution including compressed knowledge mining and integration Our solution is able to transfer high quality knowledge via noise reduction, to model rich pairwise interactions among individual-level and cluster-level entities, and to adapt the potential inconsistent knowledge from implicit feedbacks to explicit feedbacks Furthermore, the analysis on time complexity and space complexity shows that our solution is much more efficient than the state-of-the-art method for heterogeneous feedbacks Extensive empirical studies on two large data sets show that our solution is significantly better than the state-of-the-art non-transfer learning method wrt recommendation accuracy, and is much more efficient than that of leveraging the raw implicit examinations directly instead of compressed knowledge wrt CPU time and memory usage Hence, our CKT-FM strikes a good balance between effectiveness and efficiency of knowledge transfer in HCR

Congfu Xu

Papers

Adaptive Bayesian personalized ranking for heterogeneous implicit feedbacks

Sensor deployment optimization for detecting maneuvering targets

Regularized Discriminant Analysis, Ridge Regression and Beyond

Systems information in set pair analysis and its applications

Compressed knowledge transfer via factorization machine for heterogeneous collaborative recommendation