527 articles related to imbalanced data and rare events are reviewed.Viewing reviewed papers from both technical and practical perspectives.Summarizing existing methods and corresponding statistics by a new taxonomy idea.Categorizing 162 application papers into 13 domains and giving introduction.Some opening questions are discussed at the end of this manuscript. Rare events, especially those that could potentially negatively impact society, often require humans decision-making responses. Detecting rare events can be viewed as a prediction task in data mining and machine learning communities. As these events are rarely observed in daily life, the prediction task suffers from a lack of balanced data. In this paper, we provide an in depth review of rare event detection from an imbalanced learning perspective. Five hundred and seventeen related papers that have been published in the past decade were collected for the study. The initial statistics suggested that rare events detection and imbalanced learning are concerned across a wide range of research areas from management science to engineering. We reviewed all collected papers from both a technical and a practical point of view. Modeling methods discussed include techniques such as data preprocessing, classification algorithms and model evaluation. For applications, we first provide a comprehensive taxonomy of the existing application domains of imbalanced learning, and then we detail the applications for each category. Finally, some suggestions from the reviewed papers are incorporated with our experiences and judgments to offer further research directions for the imbalanced learning and rare event detection fields.

Learning from class-imbalanced data

This paper presents a review of - and classification scheme for - the literature on the application of data mining techniques for the detection of financial fraud. Although financial fraud detection (FFD) is an emerging topic of great importance, a comprehensive literature review of the subject has yet to be carried out. This paper thus represents the first systematic, identifiable and comprehensive academic literature review of the data mining techniques that have been applied to FFD. 49 journal articles on the subject published between 1997 and 2008 was analyzed and classified into four categories of financial fraud (bank fraud, insurance fraud, securities and commodities fraud, and other related financial fraud) and six classes of data mining techniques (classification, regression, clustering, prediction, outlier detection, and visualization). The findings of this review clearly show that data mining techniques have been applied most extensively to the detection of insurance fraud, although corporate fraud and credit card fraud have also attracted a great deal of attention in recent years. In contrast, we find a distinct lack of research on mortgage fraud, money laundering, and securities and commodities fraud. The main data mining techniques used for FFD are logistic models, neural networks, the Bayesian belief network, and decision trees, all of which provide primary solutions to the problems inherent in the detection and classification of fraudulent data. This paper also addresses the gaps between FFD and the needs of the industry to encourage additional research on neglected topics, and concludes with several suggestions for further FFD research.

/pdf/the-application-of-data-mining-techniques-in-financial-fraud-2xy182g2eu.pdf

The application of data mining techniques in financial fraud detection: A classification framework and an academic review of literature

Credit card fraud is a serious and growing problem. While predictive models for credit card fraud detection are in active use in practice, reported studies on the use of data mining approaches for credit card fraud detection are relatively few, possibly due to the lack of available data for research. This paper evaluates two advanced data mining approaches, support vector machines and random forests, together with the well-known logistic regression, as part of an attempt to better detect (and thus control and prosecute) credit card fraud. The study is based on real-life data of transactions from an international credit card operation.

http://didawiki.cli.di.unipi.it/lib/exe/fetch.php/dm/bankingfraude.pdf

Data mining for credit card fraud: A comparative study

Fraud detection system

A survey of anomaly detection techniques in financial domain

Due to a rapid advancement in the electronic commerce technology, the use of credit cards has dramatically increased. As credit card becomes the most popular mode of payment for both online as well as regular purchase, cases of fraud associated with it are also rising. In this paper, we model the sequence of operations in credit card transaction processing using a hidden Markov model (HMM) and show how it can be used for the detection of frauds. An HMM is initially trained with the normal behavior of a cardholder. If an incoming credit card transaction is not accepted by the trained HMM with sufficiently high probability, it is considered to be fraudulent. At the same time, we try to ensure that genuine transactions are not rejected. We present detailed experimental results to show the effectiveness of our approach and compare it with other techniques available in the literature.

Credit Card Fraud Detection Using Hidden Markov Model

http://didawiki.di.unipi.it/lib/exe/fetch.php/dm/creditcardfraudedetection-main.pdf

Credit card fraud detection: A fusion approach using Dempster-Shafer theory and Bayesian learning

A phenomenal growth in the number of credit card transactions, especially for online purchases, has recently led to a substantial rise in fraudulent activities. Implementation of efficient fraud detection systems has thus become imperative for all credit card issuing banks to minimize their losses. In real life, fraudulent transactions are interspersed with genuine transactions and simple pattern matching is not often sufficient to detect them accurately. Thus, there is a need for combining both anomaly detection as well as misuse detection techniques. In this paper, we propose to use two-stage sequence alignment in which a profile analyzer (PA) first determines the similarity of an incoming sequence of transactions on a given credit card with the genuine cardholder's past spending sequences. The unusual transactions traced by the profile analyzer are next passed on to a deviation analyzer (DA) for possible alignment with past fraudulent behavior. The final decision about the nature of a transaction is taken on the basis of the observations by these two analyzers. In order to achieve online response time for both PA and DA, we suggest a new approach for combining two sequence alignment algorithms BLAST and SSAHA.

BLAST-SSAHA Hybridization for Credit Card Fraud Detection

A phenomenal growth in the number of credit card transactions, especially for on-line purchases, has also led to a substantial rise in fraudulent activities Implementation of efficient fraud detection systems has thus become imperative for all credit card companies in order to minimize their losses In real life, fraudulent transactions could be interspersed with genuine transactions and simple pattern matching techniques are not often sufficient to detect the fraudulent transactions efficiently In this paper, we propose a hybrid approach in which anomaly detection and misuse detection models are combined Sequence alignment is used to determine similarity of an incoming sequence of transactions to both a genuine card holder's sequence as well as to sequences generated by a validated fraud model The scores from these two stages are combined to determine if a transaction is genuine or not We use stochastic models for studying the performance of the system

Two-Stage credit card fraud detection using sequence alignment

A phenomenal growth in the number of credit card transactions, especially for on-line purchases, has also led to a substantial rise in fraudulent activities. Implementation of efficient fraud detection systems has thus become imperative for all credit card companies in order to minimize their losses. In real life, fraudulent transactions could be interspersed with genuine transactions and simple pattern matching techniques are not often sufficient to detect the fraudulent transactions efficiently. In this paper, we propose a hybrid approach in which anomaly detection and misuse detection models are combined. Sequence alignment is used to determine similarity of an incoming sequence of transactions to both a genuine card holder's sequence as well as to sequences generated by a validated fraud model. The scores from these two stages are combined to determine if a transaction is genuine or not. We use stochastic models for studying the performance of the system.

Amlan Kundu

Papers

Credit Card Fraud Detection Using Hidden Markov Model

Credit card fraud detection: A fusion approach using Dempster-Shafer theory and Bayesian learning

BLAST-SSAHA Hybridization for Credit Card Fraud Detection

Two-Stage credit card fraud detection using sequence alignment

Two-stage credit card fraud detection using sequence alignment