With the continuous expansion of data availability in many large-scale, complex, and networked systems, such as surveillance, security, Internet, and finance, it becomes critical to advance the fundamental understanding of knowledge discovery and analysis from raw data to support decision-making processes. Although existing knowledge discovery and data engineering techniques have shown great success in many real-world applications, the problem of learning from imbalanced data (the imbalanced learning problem) is a relatively new challenge that has attracted growing attention from both academia and industry. The imbalanced learning problem is concerned with the performance of learning algorithms in the presence of underrepresented data and severe class distribution skews. Due to the inherent complex characteristics of imbalanced data sets, learning from such data requires new understandings, principles, algorithms, and tools to transform vast amounts of raw data efficiently into information and knowledge representation. In this paper, we provide a comprehensive review of the development of research in learning from imbalanced data. Our focus is to provide a critical review of the nature of the problem, the state-of-the-art technologies, and the current assessment metrics used to evaluate learning performance under the imbalanced learning scenario. Furthermore, in order to stimulate future research in this field, we also highlight the major opportunities and challenges, as well as potential important research directions for learning from imbalanced data.

Learning from Imbalanced Data

A systematic study of the class imbalance problem in convolutional neural networks

Machine learning for medical diagnosis: history, state of the art and perspective

This paper studies empirically the effect of sampling and threshold-moving in training cost-sensitive neural networks. Both oversampling and undersampling are considered. These techniques modify the distribution of the training data such that the costs of the examples are conveyed explicitly by the appearances of the examples. Threshold-moving tries to move the output threshold toward inexpensive classes such that examples with higher costs become harder to be misclassified. Moreover, hard-ensemble and soft-ensemble, i.e., the combination of above techniques via hard or soft voting schemes, are also tested. Twenty-one UCl data sets with three types of cost matrices and a real-world cost-sensitive data set are used in the empirical study. The results suggest that cost-sensitive learning with multiclass tasks is more difficult than with two-class tasks, and a higher degree of class imbalance may increase the difficulty. It also reveals that almost all the techniques are effective on two-class tasks, while most are ineffective and even may cause negative effect on multiclass tasks. Overall, threshold-moving and soft-ensemble are relatively good choices in training cost-sensitive neural networks. The empirical study also suggests that some methods that have been believed to be effective in addressing the class imbalance problem may, in fact, only be effective on learning with imbalanced two-class data sets.

/pdf/training-cost-sensitive-neural-networks-with-methods-n2m3djdnmc.pdf

Training cost-sensitive neural networks with methods addressing the class imbalance problem

In the era of big data, transformation of biomedical big data into valuable knowledge has been one of the most important challenges in bioinformatics. Deep learning has advanced rapidly since the early 2000s and now demonstrates state-of-the-art performance in various fields. Accordingly, application of deep learning in bioinformatics to gain insight from data has been emphasized in both academia and industry. Here, we review deep learning in bioinformatics, presenting examples of current research. To provide a useful and comprehensive perspective, we categorize research both by the bioinformatics domain (i.e. omics, biomedical imaging, biomedical signal processing) and deep learning architecture (i.e. deep neural networks, convolutional neural networks, recurrent neural networks, emergent architectures) and present brief descriptions of each study. Additionally, we discuss theoretical and practical issues of deep learning in bioinformatics and suggest future research directions. We believe that this review will provide valuable insights and serve as a starting point for researchers to apply deep learning approaches in their bioinformatics studies.

Deep learning in bioinformatics

In the usual setting of Machine Learning, classifiers are typically evaluated by estimating their error rate (or equi valently, the classification accuracy) on the test data. However, this mak es sense only if all errors have equal (uniform) costs. When the costs of er- rors differ between each other, the classifiers should be eva luated by comparing the total costs of the errors. Classifiers are typically designed to minimize the number of errors (incorrect classifications) made. When misclassification c osts vary between classes, this approach is not suitable. In this case the total misclassification cost should be minimized. In Machine Learning, only little work for dealing with non- uniform misclassification costs has been done. This paper pr esents a few different approaches for cost-sensitive modifications of the back- propagation learning algorithm for multilayered feedforw ard neural networks . The described approaches are thoroughly tested and eval- uated on several standard benchmark domains.

Cost-Sensitive Learning with Neural Networks.

Introduction Learning and intelligence Machine learning basics Knowledge representation Learning as search Attribute quality measures Data pre-processing Constructive induction Symbolic learning Statistical learning Artificial neural networks Cluster analysis Learning theory Computational learning theory Definitions References and index.

Machine Learning and Data Mining: Introduction to Principles and Algorithms

In the past decades Machine Learning algorithms have been successfully used in numerous classification problems. While they usually significantly outperform domain experts (in terms of classification accuracy or otherwise), they are mostly not being used in practice. A plausible reason for this is that it is difficult to obtain an unbiased estimation of a single classification's reliability. In the paper we propose a general transductive method for estimation of classification's reliability on single examples that is independent of the applied Machine Learning algorithm. We compare our method with existing approaches and discuss its advantages. We perform extensive testing on 14 domains and 6 Machine Learning algorithms and show that our approach can frequently yield more than 100% improvement in reliability estimation performance.

/pdf/reliable-classifications-with-machine-learning-4vm543k05q.pdf

Reliable Classifications with Machine Learning

The diagnostic process of coronary artery disease (CAD) consists of evaluation of symptoms and signs of the disease and ECG at rest, ECG during exercise, myocardial perfusion scintigraphy (MPS) and coronary angiography. Machine Learning (ML) can use all particular data in interpretation of result. The authors' goal was to predict in a group of 327 patients the results of coronary angiography obtained by ML method and compare them with the results of MPS as the highest step in the classical diagnostic procedure. The Naive Bayesian Classifier as one of the ML methods was applied. The sensitivity of MPS was 0.83 and specificity 0.85. The post-test probability for CAD was 0.75 for positive results and 0.43 for negative ones. With application of ML the authors achieved sensitivity 0.89, specificity 0.88 and the post-test probability 0.90 for positive and 0.25 for negative results.

Machine learning improves the accuracy of coronary artery disease diagnostic methods

Ischaemic heart disease is one of the world's most important causes of mortality, so improvements and rationalization of diagnostic procedures would be very useful. The four diagnostic levels consist of evaluation of signs and symptoms of the disease and ECG (electrocardiogram) at rest, sequential ECG testing during the controlled exercise, myocardial scintigraphy and finally coronary angiography. The diagnostic process is stepwise and the results are interpreted hierarchically, i.e, the next step is necessary only if the results of the former are inconclusive. Because suggestibility is possible, the results of each step are interpreted individually and only the results of the highest step are valid. On the other hand, machine learning methods may be capable of objective interpretation of all available results for the same patient and in this way increase the diagnostic accuracy, sensitivity and specificity of each step. In the usual setting, the machine learning algorithms are tuned to maximize classification accuracy. In our case, the sensitivity and specificity were much more important, so we generalized the algorithms to take in account the variable misclassification costs. The costs can be tuned in order to bias the algorithms towards higher sensitivity or specificity. We conducted many experiments with four learning algorithms and different variations of our dataset (327 patients with completed diagnostic procedures). Our results show that improvements using machine learning techniques are reasonable and might find good use in practice.

Matjaz Kukar

Papers

Cost-Sensitive Learning with Neural Networks.

Machine Learning and Data Mining: Introduction to Principles and Algorithms

Reliable Classifications with Machine Learning

Machine learning improves the accuracy of coronary artery disease diagnostic methods

An application of machine learning in the diagnosis of ischaemic heart disease