This paper introduces ICET, a new algorithm for cost-sensitive classification. ICET uses a genetic algorithm to evolve a population of biases for a decision tree induction algorithm. The fitness function of the genetic algorithm is the average cost of classification when using the decision tree, including both the costs of tests (features, measurements) and the costs of classification errors. ICET is compared here with three other algorithms for cost-sensitive classification - EG2, CS-ID3, and IDX - and also with C4.5, which classifies without regard to cost. The five algorithms are evaluated empirically on five real-world medical datasets. Three sets of experiments are performed. The first set examines the baseline performance of the five algorithms on the five datasets and establishes that ICET performs significantly better than its competitors. The second set tests the robustness of ICET under a variety of conditions and shows that ICET maintains its advantage. The third set looks at ICET's search in bias space and discovers a way to improve the search.

Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm

Inductive concept learning is the task of learning to assign cases to a discrete set of classes. In real-world applications of concept learning, there are many different types of cost involved. The majority of the machine learning literature ignores all types of cost (unless accuracy is interpreted as a type of cost measure). A few papers have investigated the cost of misclassification errors. Very few papers have examined the many other types of cost. In this paper, we attempt to create a taxonomy of the different types of cost that are involved in inductive concept learning. This taxonomy may help to organize the literature on cost-sensitive learning. We hope that it will inspire researchers to investigate all types of cost in inductive concept learning in more depth.

/pdf/types-of-cost-in-inductive-concept-learning-1m53zs204m.pdf

Types of cost in inductive concept learning

This paper describes work in the StatLog project comparing classification algorithms on large real-world problems The algorithms compared were from symbolic learning (CART C45, NewID, AC2,ITrule, Cal5, CN2), statistics (Naive Bayes, k-nearest neighbor, kernel density, linear discriminant, quadratic discriminant, logistic regression, projection pursuit, Bayesian networks), and neural networks (backpropagation, radial basis functions) Twelve datasets were used: five from image analysis, three from medicine, and two each from engineering and finance We found that which algorithm performed best depended critically on the data set investigated We therefore developed a set of data set descriptors to help decide which algorithms are suited to particular data sets For example, data sets with extreme distributions (skew > l and kurtosis > 7) and with many binary/categorical attributes (>38%) tend to favor symbolic learning algorithms We suggest how classification algorithms can be extended in a number of d

Statlog: comparison of classification algorithms on large real-world problems

We present novel optimization-based classification models that are general purpose and suitable for developing predictive rules for large heterogeneous biological and medical data sets. Our predictive model simultaneously incorporates (1) the ability to classify any number of distinct groups; (2) the ability to incorporate heterogeneous types of attributes as input; (3) a high-dimensional data transformation that eliminates noise and errors in biological data; (4) the ability to incorporate constraints to limit the rate of misclassification, and a reserved-judgment region that provides a safeguard against over-training (which tends to lead to high misclassification rates from the resulting predictive rule); and (5) successive multi-stage classification capability to handle data points placed in the reserved-judgment region. To illustrate the power and flexibility of the classification model and solution engine, and its multi-group prediction capability, application of the predictive model to a broad class of biological and medical problems is described. Applications include: the differential diagnosis of the type of erythemato-squamous diseases; predicting presence/absence of heart disease; genomic analysis and prediction of aberrant CpG island meythlation in human cancer; discriminant analysis of motility and morphology data in human lung carcinoma; prediction of ultrasonic cell disruption for drug delivery; identification of tumor shape and volume in treatment of sarcoma; discriminant analysis of biomarkers for prediction of early atherosclerois; fingerprinting of native and angiogenic microvascular networks for early diagnosis of diabetes, aging, macular degeneracy and tumor metastasis; prediction of protein localization sites; and pattern recognition of satellite images in classification of soil types. In all these applications, the predictive model yields correct classification rates ranging from 80 to 100%. This provides motivation for pursuing its use as a medical diagnostic, monitoring and decision-making tool.

https://link.springer.com/content/pdf/10.1007%2Fs10439-007-9317-7.pdf

Large-Scale Optimization-Based Classification Models in Medicine and Biology

Potential methods in pattern recognition

SUMMARY Allocation problems can involve elements whose original population is not conclusively indicated by the observation. These elements are called 'cases of doubt'. In this paper a decision-theoretical model for allocation problems admitting cases of doubt is developed. The sampling situation is discussed both from a distribution-free point of view and for multinormal distributions. The theory is illustrated with an example from medical dif- ferential diagnosis. Consider a population A of elements that can be split into two populations A1 and A2, such that A = A1 U A2 and Al n A2 = 0. The frequency of occurrence of the two populations will be specified in the a priori probabilities nr(A1) and Ir(A2). For each element we measure a p-dimensional vector variable x = (xl, ..., xp)'. The random variation of x in the two popula- tions will be described by the distribution functions F(xJAj) and F(xJA2). Suppose that we have an element randomly selected from A, but we do not know whether it comes from A1 or A2. The allocation problem we now consider is how to reach a decision about the origin of this element on the basis of its observation x. There is an extensive literature on this problem if one is forced to allocate the element to either A1 or A2. If one is not forced to allocate to either A1 or A2, one would like to introduce for inconclusive observations an 'allocation of doubt'. In this situation, for every possible observation x, we must choose one of three allocations:

J. Hermans

Papers

Cases of doubt in allocation problems