Data Mining Practical Machine Learning Tools and Techniques

Peer-to-peer (P2P) lending demands effective and explainable credit risk models. Typical machine learning algorithms offer high prediction performance, but most of them lack explanatory power. However, this deficiency can be solved with the help of the explainability tools proposed in the last few years, such as the SHAP values. In this work, we assess the well-known logistic regression model and several machine learning algorithms for granting scoring in P2P lending. The comparison reveals that the machine learning alternative is superior in terms of not only classification performance but also explainability. More precisely, the SHAP values reveal that machine learning algorithms can reflect dispersion, nonlinearity and structural breaks in the relationships between each feature and the target variable. Our results demonstrate that is possible to have machine learning credit scoring models be both accurate and transparent. Such models provide the trust that the industry, regulators and end-users demand in P2P lending and may lead to a wider adoption of machine learning in this and other risk assessment applications where explainability is required.

Explainability of a Machine Learning Granting Scoring Model in Peer-to-Peer Lending

Peer to Peer (P2P) Lending Problems and Potential Solutions: A Systematic Literature Review

This paper presents a benchmark of supervised Automated Machine Learning (AutoML) tools. Firstly, we analyze the characteristics of eight recent open-source AutoML tools (Auto-Keras, Auto-PyTorch, Auto-Sklearn, AutoGluon, H2O AutoML, rminer, TPOT and TransmogrifAI) and describe twelve popular OpenML datasets that were used in the benchmark (divided into regression, binary and multi-class classification tasks). Then, we perform a comparison study with hundreds of computational experiments based on three scenarios: General Machine Learning (GML), Deep Learning (DL) and XGBoost (XGB). To select the best tool, we used a lexicographic approach, considering first the average prediction score for each task and then the computational effort. The best predictive results were achieved for GML, which were further compared with the best OpenML public results. Overall, the best GML AutoML tools obtained competitive results, outperforming the best OpenML models in five datasets. These results confirm the potential of the general-purpose AutoML tools to fully automate the Machine Learning (ML) algorithm selection and tuning.

/pdf/a-comparison-of-automl-tools-for-machine-learning-deep-33myfrok54.pdf

A Comparison of AutoML Tools for Machine Learning, Deep Learning and XGBoost

Learning nonstationary data streams has been well studied in recent years. However, most of the researches assume that the class imbalance of data streams is relatively balanced. Only a few approaches tackle the joint issue of concept drift and class imbalance due to its complexity. Meanwhile, the existing chunk ensembles for classifying imbalanced nonstationary data streams always need to store previous data, which consumes plenty of memory usage. To overcome these issues, we propose a chunk-based incremental ensemble algorithm called Dynamic Updated Ensemble (DUE) for learning imbalanced data streams with concept drift. Compared to the existing techniques, its merits are five-fold: (1) it learns one chunk at a time without requiring access to previous data; (2) it emphasizes misclassified examples in the model update procedure; (3) it can timely react to multiple kinds of concept drifts; (4) it can adapt to the new condition when switching majority class to minority class; (5) it keeps a limited number of classifiers to ensure high efficiency. Experiments on synthetic and real datasets demonstrate the effectiveness of DUE in learning nonstationary imbalanced data streams.

Incremental learning imbalanced data streams with concept drift: The dynamic updated ensemble algorithm

The large volume of data generated by computer networks, smartphones, wearables and a wide range of sensors, which produce real-time data, are only useful if they can be efficiently processed so that individuals can make timely decisions based on them. In this context, machine learning techniques are widely used. While it performs better than humans in such tasks, every machine learning algorithm has a certain intrinsic bias, which means they assume that the data have specific characteristics, such as having a balanced distribution between classes. As many real-world applications present imbalanced traits in their data, this topic is gaining repercussion over time. In this work, we present the Adaptive Random Forest with Resampling (ARF RE ), which is a classifier designed to deal with imbalanced datasets. ARF RE resample the instances based on the current class label distribution. We show through a set of extensive experiments on seven datasets that the proposed method can considerably improve the performance of the minority class(es) while avoiding degrading the performance in the majority class. On top of that, ARF RE is more efficient regarding execution time in comparison to the standard ARF algorithm.

Adaptive Random Forests with Resampling for Imbalanced data Streams

Data stream mining is a hot topic in the machine learning community that tackles the problem of learning and updating predictive models as new data becomes available over time. Even though several new methods are proposed every year, most focus on the classification task and overlook the regression task. In this paper, we propose an adaptation to the Adaptive Random Forest so that it can handle regression tasks, namely ARF-Reg. ARF-Reg is empirically evaluated and compared to the state-of-the-art data stream regression algorithms, thus highlighting its applicability in different data stream scenarios.

Adaptive random forests for data stream regression

Peer-to-peer (P2P) lending is a global trend of financial markets that allow individuals to obtain and concede loans without having financial institutions as a strong proxy. As many real-world applications, P2P lending presents an imbalanced characteristic, where the number of creditworthy loan requests is much larger than the number of non-creditworthy ones. In this work, we wrangle a real-world P2P lending data set from Lending Club, containing a large amount of data gathered from 2007 up to 2016. We analyze how supervised classification models and techniques to handle class imbalance impact creditworthiness prediction rates. Ensembles, cost-sensitive and sampling methods are combined and evaluated along logistic regression, decision tree, and bayesian learning schemes. Results show that, in average, sampling techniques outperform ensembles and cost sensitive approaches.

Improving Credit Risk Prediction in Online Peer-to-Peer (P2P) Lending Using Imbalanced Learning Techniques

This work was executed under the project IR-MDA - Intelligent Risk Management for the Digital Age, Individual Project, NUP: POCI-01-0247-FEDER-038526, co-funded by the Incentive Systemfor Research and Technological Development, fromthe Thematic Operational Program Competitivenessof the national framework program - Portugal2020.

/pdf/an-automated-and-distributed-machine-learning-framework-for-2us41gsnp5.pdf

Luís Ferreira

Papers

A Comparison of AutoML Tools for Machine Learning, Deep Learning and XGBoost

Adaptive Random Forests with Resampling for Imbalanced data Streams

Adaptive random forests for data stream regression

Improving Credit Risk Prediction in Online Peer-to-Peer (P2P) Lending Using Imbalanced Learning Techniques

An automated and distributed machine learning framework for telecommunications risk management