A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research.

/pdf/a-survey-on-transfer-learning-1hjmu3otql.pdf

A Survey on Transfer Learning

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Data Mining - Concepts and Techniques.

Data Mining Practical Machine Learning Tools and Techniques

https://uwspace.uwaterloo.ca/bitstream/handle/10012/3000/thesis.pdf;sequence=1

Cost-sensitive boosting for classification of imbalanced data

Associative classification is a promising technique to build accurate classifiers. However, in large or correlated data sets, association rule mining may yield huge rule sets. Hence, several pruning techniques have been proposed to select a small subset of high-quality rules. Since the availability of a "rich" rule set may improve the accuracy of the classifier, we argue that rule pruning should be reduced to a minimum. The L3 associative classifier is built by means of a lazy pruning technique that discards exclusively rules that only misclassify training data. The classification of unlabeled data is performed in two steps. A small subset of high-quality rules is first considered. When this set is not able to classify the data, a larger rule set is exploited. This second set includes rules usually discarded by previous approaches. To cope with the need of mining large rule sets and to efficiently use them for classification, a compact form is proposed to represent a complete rule set in a space-efficient way and without information loss. An extensive experimental evaluation on real and synthetic data sets shows that L:i improves the classification accuracy with respect to previous approaches.

A Lazy Approach to Associative Classification

Associative classification is a promising technique for the generation of highly precise classifiers. Previous works propose several clever techniques to prune the huge set of generated rules, with the twofold aim of selecting a small set of high quality rules, and reducing the chance of overfitting. In this paper, we argue that pruning should be reduced to a minimum and that the availability of a large rule base may improve the precision of the classifier without affecting its performance. In L/sup 3/ (Live and Let Live), a new algorithm for associative classification, a lazy pruning technique iteratively discards all rules that only yield wrong case classifications. Classification is performed in two steps. Initially, rules which have already correctly classified at least one training case, sorted by confidence, are considered If the case is still unclassified, the remaining rules (unused during the training phase) are considered, again sorted by confidence. Extensive experiments on 26 databases from the UCI machine learning database repository show that L/sup 3/ improves the classification precision with respect to previous approaches.

A lazy approach to pruning classification rules

Frequent weighted itemsets represent correlations frequently holding in data in which items may weight differently. However, in some contexts, e.g., when the need is to minimize a certain cost function, discovering rare data correlations is more interesting than mining frequent ones. This paper tackles the issue of discovering rare and weighted itemsets, i.e., the infrequent weighted itemset (IWI) mining problem. Two novel quality measures are proposed to drive the IWI mining process. Furthermore, two algorithms that perform IWI and Minimal IWI mining efficiently, driven by the proposed measures, are presented. Experimental results show efficiency and effectiveness of the proposed approach.

/pdf/infrequent-weighted-itemset-mining-using-frequent-pattern-3unebmpwuu.pdf

Infrequent Weighted Itemset Mining Using Frequent Pattern Growth

https://iris.polito.it/bitstream/11583/2460918/1/2460918_draft.pdf

Generalized association rule mining with constraints

Associative classification is a well-known technique for structured data classification. Most previous works on associative classification use support based pruning for rule extraction, and usually set the threshold value to 1%. This threshold allows rule extraction to be tractable and on the average yields a good accuracy. We believe that this threshold may be not accurate in some cases, since the class distribution in the dataset is not taken into account. In this paper we investigate the effect of support threshold on classification accuracy. Lower support thresholds are often unfeasible with current extraction algorithms, or may cause the generation of a huge rule set. To observe the effect of varying the support threshold, we first propose a compact form to encode a complete rule set. We then develop a new classifier, named L3G, based on the compact form. Taking advantage of the compact form, the classifier can be built also with rather low support rules. We ran a variety of experiments with different support thresholds on datasets from the UCI machine learning database repository. The experiments showed that the optimal accuracy is obtained for variable threshold values, sometime lower than 1%.

Paolo Garza

Papers

A Lazy Approach to Associative Classification

A lazy approach to pruning classification rules

Infrequent Weighted Itemset Mining Using Frequent Pattern Growth

Generalized association rule mining with constraints

On support thresholds in associative classification