An approach to the construction of classifiers from imbalanced datasets is described. A dataset is imbalanced if the classification categories are not approximately equally represented. Often real-world data sets are predominately composed of "normal" examples with only a small percentage of "abnormal" or "interesting" examples. It is also the case that the cost of misclassifying an abnormal (interesting) example as a normal example is often much higher than the cost of the reverse error. Under-sampling of the majority (normal) class has been proposed as a good means of increasing the sensitivity of a classifier to the minority class. This paper shows that a combination of our method of oversampling the minority (abnormal)cla ss and under-sampling the majority (normal) class can achieve better classifier performance (in ROC space)tha n only under-sampling the majority class. This paper also shows that a combination of our method of over-sampling the minority class and under-sampling the majority class can achieve better classifier performance (in ROC space)t han varying the loss ratios in Ripper or class priors in Naive Bayes. Our method of over-sampling the minority class involves creating synthetic minority class examples. Experiments are performed using C4.5, Ripper and a Naive Bayes classifier. The method is evaluated using the area under the Receiver Operating Characteristic curve (AUC)and the ROC convex hull strategy.

/pdf/smote-synthetic-minority-over-sampling-technique-5ap1gi5ups.pdf

SMOTE: synthetic minority over-sampling technique

An approach to the construction of classifiers from imbalanced datasets is described. A dataset is imbalanced if the classification categories are not approximately equally represented. Often real-world data sets are predominately composed of "normal" examples with only a small percentage of "abnormal" or "interesting" examples. It is also the case that the cost of misclassifying an abnormal (interesting) example as a normal example is often much higher than the cost of the reverse error. Under-sampling of the majority (normal) class has been proposed as a good means of increasing the sensitivity of a classifier to the minority class. This paper shows that a combination of our method of over-sampling the minority (abnormal) class and under-sampling the majority (normal) class can achieve better classifier performance (in ROC space) than only under-sampling the majority class. This paper also shows that a combination of our method of over-sampling the minority class and under-sampling the majority class can achieve better classifier performance (in ROC space) than varying the loss ratios in Ripper or class priors in Naive Bayes. Our method of over-sampling the minority class involves creating synthetic minority class examples. Experiments are performed using C4.5, Ripper and a Naive Bayes classifier. The method is evaluated using the area under the Receiver Operating Characteristic curve (AUC) and the ROC convex hull strategy.

/pdf/smote-synthetic-minority-over-sampling-technique-3s7jks70tk.pdf

SMOTE: Synthetic Minority Over-sampling Technique

This paper presents an overview of the field of recommender systems and describes the current generation of recommendation methods that are usually classified into the following three main categories: content-based, collaborative, and hybrid recommendation approaches. This paper also describes various limitations of current recommendation methods and discusses possible extensions that can improve recommendation capabilities and make recommender systems applicable to an even broader range of applications. These extensions include, among others, an improvement of understanding of users and items, incorporation of the contextual information into the recommendation process, support for multicriteria ratings, and a provision of more flexible and less intrusive types of recommendations.

/pdf/toward-the-next-generation-of-recommender-systems-a-survey-4wwsv77rns.pdf

Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions

Many existing rule learning systems are computationally expensive on large noisy datasets. In this paper we evaluate the recently-proposed rule learning algorithm IREP on a large and diverse collection of benchmark problems. We show that while IREP is extremely efficient, it frequently gives error rates higher than those of C4.5 and C4.5rules. We then propose a number of modifications resulting in an algorithm RIPPERk that is very competitive with C4.5rules with respect to error rates, but much more efficient on large samples. RIPPERk obtains error rates lower than or equivalent to C4.5rules on 22 of 37 benchmark problems, scales nearly linearly with the number of training examples, and can efficiently process noisy datasets containing hundreds of thousands of examples.

Fast effective rule induction

In machine learning problems, differences in prior class probabilities -- or class imbalances -- have been reported to hinder the performance of some standard classifiers, such as decision trees. This paper presents a systematic study aimed at answering three different questions. First, we attempt to understand the nature of the class imbalance problem by establishing a relationship between concept complexity, size of the training set and class imbalance level. Second, we discuss several basic re-sampling or cost-modifying methods previously proposed to deal with the class imbalance problem and compare their effectiveness. The results obtained by such methods on artificial domains are linked to results in real-world domains. Finally, we investigate the assumption that the class imbalance problem does not only affect decision tree systems but also affects other classification systems such as Neural Networks and Support Vector Machines.

The class imbalance problem: A systematic study

We explore algorithms for learning classification procedures that attempt to minimize the cost of misclassifying examples. First, we consider inductive learning of classification rules. The Reduced Cost Ordering algorithm, a new method for creating a decision list (i.e., an ordered set of rules) is described and compared to a variety of inductive learning approaches. Next, we describe approaches that attempt to minimize costs while avoiding overfitting, and introduce the Clause Prefix method for pruning decision lists. Finally, we consider reducing misclassification costs when a prior domain theory is available.

Reducing misclassification costs

The World Web Wide gives unprecedented access to Newspapers, magazines, shopping catalogs, restaurant guides, and classified ads and other types of information. All this information, however, used to be accessible only while users are tethered to a computer at home or in an office. Wireless data and voice access to this vast store allows unprecedented access to information from any location at any time.

/pdf/adaptive-interfaces-for-ubiquitous-web-access-4jky1f0zb8.pdf

Adaptive interfaces for ubiquitous web access

We describe a new approach to integrating explanation-based and empirical learning methods for learning relational concepts. The approach uses an information-based heuristic to evaluate components of a hypothesis that are proposed either by explanation-based or empirical learning methods. Providing domain knowledge to the integrated system can decrease the amount of search required during learning and increase the accuracy of learned concepts, even when the domain knowledge is incorrect and incomplete.

A knowledge-intensive approach to learning relational concepts

We discuss the types of noise that may occur in relational learning systems and describe two approaches to addressing noise in a relational concept learning algorithm. We then evaluate each approach experimentally.

/pdf/an-investigation-of-noise-tolerant-relational-concept-4q19khiatk.pdf

An investigation of noise-tolerant relational concept learning algorithms

https://www.ics.uci.edu/~pazzani/Publications/detecting.pdf

Clifford Brunk

Papers

Reducing misclassification costs

Adaptive interfaces for ubiquitous web access

A knowledge-intensive approach to learning relational concepts

An investigation of noise-tolerant relational concept learning algorithms

Detecting and correcting errors in rule-based expert systems: an integration of empirical and explanation-based learning