Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

527 articles related to imbalanced data and rare events are reviewed.Viewing reviewed papers from both technical and practical perspectives.Summarizing existing methods and corresponding statistics by a new taxonomy idea.Categorizing 162 application papers into 13 domains and giving introduction.Some opening questions are discussed at the end of this manuscript. Rare events, especially those that could potentially negatively impact society, often require humans decision-making responses. Detecting rare events can be viewed as a prediction task in data mining and machine learning communities. As these events are rarely observed in daily life, the prediction task suffers from a lack of balanced data. In this paper, we provide an in depth review of rare event detection from an imbalanced learning perspective. Five hundred and seventeen related papers that have been published in the past decade were collected for the study. The initial statistics suggested that rare events detection and imbalanced learning are concerned across a wide range of research areas from management science to engineering. We reviewed all collected papers from both a technical and a practical point of view. Modeling methods discussed include techniques such as data preprocessing, classification algorithms and model evaluation. For applications, we first provide a comprehensive taxonomy of the existing application domains of imbalanced learning, and then we detail the applications for each category. Finally, some suggestions from the reviewed papers are incorporated with our experiences and judgments to offer further research directions for the imbalanced learning and rare event detection fields.

Learning from class-imbalanced data

Trends in extreme learning machines

Credit scoring is an effective tool for banks to properly guide decision profitably on granting loans. Ensemble methods, which according to their structures can be divided into parallel and sequential ensembles, have been recently developed in the credit scoring domain. These methods have proven their superiority in discriminating borrowers accurately. However, among the ensemble models, little consideration has been provided to the following: (1) highlighting the hyper-parameter tuning of base learner despite being critical to well-performed ensemble models; (2) building sequential models (i.e., boosting, as most have focused on developing the same or different algorithms in parallel); and (3) focusing on the comprehensibility of models. This paper aims to propose a sequential ensemble credit scoring model based on a variant of gradient boosting machine (i.e., extreme gradient boosting (XGBoost)). The model mainly comprises three steps. First, data pre-processing is employed to scale the data and handle missing values. Second, a model-based feature selection system based on the relative feature importance scores is utilized to remove redundant variables. Third, the hyper-parameters of XGBoost are adaptively tuned with Bayesian hyper-parameter optimization and used to train the model with selected feature subset. Several hyper-parameter optimization methods and baseline classifiers are considered as reference points in the experiment. Results demonstrate that Bayesian hyper-parameter optimization performs better than random search, grid search, and manual search. Moreover, the proposed model outperforms baseline models on average over four evaluation measures: accuracy, error rate, the area under the curve (AUC) H measure (AUC-H measure), and Brier score. The proposed model also provides feature importance scores and decision chart, which enhance the interpretability of credit scoring model.

A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring

Flood hazard risk assessment model based on random forest

The prediction of business failure is an important and challenging issue that has served as the impetus for many academic studies over the past three decades. While the efficiency of a corporation's management is generally acknowledged to be a key contributor to corporation's bankrupt, it is usually excluded from early prediction models. The objective of the study is to use efficiency as predictive variables with a proposed novel model to integrate rough set theory (RST) with support vector machines (SVM) technique to increase the accuracy of the prediction of business failure. In the proposed method (RST-SVM), data envelopment analysis (DEA) is employed as a tool to evaluate the input/output efficiency. Furthermore, by RST approach, the redundant attributes in multi-attribute information table can be reduced, which showed that the number of independent variables was reduced with no information loss, is utilized as a preprocessor to improve business failure prediction capability by SVM. The effectiveness of the methodology was verified by experiments comparing back-propagation neural networks (BPN) approach with the hybrid approach (RST-BPN). The results shows that DEA do provide valuable information in business failure predictions and the proposed RST-SVM model provides better classification results than RST-BPN model, no matter when only considering financial ratios or the model including both financial ratios and DEA.

A hybrid approach of DEA, rough set and support vector machines for business failure prediction

Detecting fraudulent financial statements (FFS) is critical in order to protect the global financial market. In recent years, FFS have begun to appear and continue to grow rapidly, which has shocked the confidence of investors and threatened the economics of entire countries. While auditors are the last line of defense to detect FFS, many auditors lack the experience and expertise to deal with the related risks. This study introduces a support vector machine-based fraud warning (SVMFW) model to reduce these risks. The model integrates sequential forward selection (SFS), support vector machine (SVM), and a classification and regression tree (CART). SFS is employed to overcome information overload problems, and the SVM technique is then used to assess the likelihood of FFS. To select the parameters of SVM models, particle swarm optimization (PSO) is applied. Finally, CART is employed to enable auditors to increase substantive testing during their audit procedures by adopting reliable, easy-to-grasp decision rules. The experiment results show that the SVMFW model can reduce unnecessary information, satisfactorily detect FFS, and provide directions for properly allocating audit resources in limited audits. The model is a promising alternative for detecting FFS caused by top management, and it can assist in both taxation and the banking system.

A support vector machine-based model for detecting top management fraud

Pre-warning of whether a corporate will fall into a decline stage in the near future is an emerging issue in financial management Improper decision-making by firms incurs a higher possibility to cause financial crisis (distress) and deteriorates the soundness of financial markets The aim of this study is to establish a novel prediction mechanism based on combining the sampling technique (synthetic minority over-sampling technique; SMOTE), feature selection ensemble (original, intersection, and union), extreme learning machine (ELM) ensemble and decision tree (DT) The proposed model - namely, the multiple extreme learning machines (MELMs) - shows promising performance under numerous assessing criteria, but one critical defect of the ensemble classifier is that it lacks comprehensibility Thus, we perform a DT as the knowledge generator to extract the inherent information from the ensemble mechanism This knowledge visualized process can assist decision makers in efficiently allocating limited financial resources and to help firms survive in an extremely competitive environment

Multiple extreme learning machines for a two-class imbalance corporate life cycle prediction

The sub-prime mortgage crisis of 2007 and the global financial tsunami of 2008 have undermined worldwide economic stability. Consequently, the timely analysis of credit risk has become more essential than ever before. The performance of early risk warning mechanisms may vary according to the criteria used and the underlying environment. This study establishes numerous criteria to assess the performance of classifiers and introduces a multiple criteria decision making method to determine suitable warning mechanisms. After undergoing these evaluations, the enhanced decision support model (EDSM), which incorporates the relevance vector machine with decision tree, is proposed. A decision tree is employed to overcome the opaque nature of the relevance vector machine; it yields knowledge as logical statements and aids in the interpretability of the forecasting results. The advantages of the EDSM involve overcoming the timeliness problem, fostering faster credit financing decisions, diminishing possible mistakes and reducing the credit analysis cost. This study also examines the feasibility of corporate transparency and the information disclosure (TD) criterion during an upturn in the economy, and finds that this procedure presents a suitable policy-relevant direction for regulators to design future measurements. Finally, this study shows that the EDSM is a promising way for investors, creditors, bankers and regulators to analyze credit rating domains.

https://isiarticles.com/bundles/Article/pre/pdf/48679.pdf

Ming-Fu Hsu

Papers

A hybrid approach of DEA, rough set and support vector machines for business failure prediction

A support vector machine-based model for detecting top management fraud

Multiple extreme learning machines for a two-class imbalance corporate life cycle prediction

Credit risk assessment and decision making by a fusion approach

An ensemble-based model for two-class imbalanced financial problem