Machine learning for email spam filtering: review, approaches and open research problems

Pervasive growth and usage of the Internet and mobile applications have expanded cyberspace. The cyberspace has become more vulnerable to automated and prolonged cyberattacks. Cyber security techniques provide enhancements in security measures to detect and react against cyberattacks. The previously used security systems are no longer sufficient because cybercriminals are smart enough to evade conventional security systems. Conventional security systems lack efficiency in detecting previously unseen and polymorphic security attacks. Machine learning (ML) techniques are playing a vital role in numerous applications of cyber security. However, despite the ongoing success, there are significant challenges in ensuring the trustworthiness of ML systems. There are incentivized malicious adversaries present in the cyberspace that are willing to game and exploit such ML vulnerabilities. This paper aims to provide a comprehensive overview of the challenges that ML techniques face in protecting cyberspace against attacks, by presenting a literature on ML techniques for cyber security including intrusion detection, spam detection, and malware detection on computer networks and mobile networks in the last decade. It also provides brief descriptions of each ML method, frequently used security datasets, essential ML tools, and evaluation metrics to evaluate a classification model. It finally discusses the challenges of using ML techniques in cyber security. This paper provides the latest extensive bibliography and the current trends of ML in cyber security.

/pdf/a-survey-on-machine-learning-techniques-for-cyber-security-i849l0zme0.pdf

A Survey on Machine Learning Techniques for Cyber Security in the Last Decade

Spam is commonly defined as unwanted e-mails and it became a global threat against e-mail users. Although, Support Vector Machine (SVM) has been commonly used in e-mail spam classification, yet the problem of high data dimensionality of the feature space due to the massive number of e-mail dataset and features still exist. To improve the limitation of SVM, reduce the computational complexity (efficiency) and enhancing the classification accuracy (effectiveness). In this study, feature selection based on one-way ANOVA F-test statistics scheme was applied to determine the most important features contributing to e-mail spam classification. This feature selection based on one-way ANOVA F-test is used to reduce the high data dimensionality of the feature space before the classification process. The experiment of the proposed scheme was carried out using spam base well- known benchmarking dataset to evaluate the feasibility of the proposed method. The comparison is achieved for different datasets, categorization algorithm and success measures. In addition, experimental results on spam base English datasets showed that the enhanced SVM (FSSVM) significantly outperforms SVM and many other recent spam classification methods for English dataset in terms of computational complexity and dimension reduction.

A Novel Feature Selection Based on One-Way ANOVA F-Test for E-Mail Spam Classification

Feature selection is a problem of global combinator ial optimization in machine learning in which subse ts of relevant features are selected to realize robust le arning models. The inclusion of irrelevant and redu ndant features in the dataset can result in poor predicti ons and high computational overhead. Thus, selectin g relevant feature subsets can help reduce the comput ational cost of feature measurement, speed up learn ing process and improve model interpretability. SVM classifier has proven inefficient in its inability to produce accurate classification results in the face of larg e e-mail dataset while it also consumes a lot of computational resources. In this study, a Genetic A lgorithm-Support Vector Machine (GA-SVM) feature selection technique is developed to optimize the SV M classification parameters, the prediction accurac y and computation time. Spam assassin dataset was used to validate the performance of the proposed syste m. The hybrid GA-SVM showed remarkable improvements over SVM in terms of classification accuracy and computation time.

/pdf/hybrid-ga-svm-for-efficient-feature-selection-in-e-mail-2uxzwsauqz.pdf

Hybrid GA-SVM for Efficient Feature Selection in E-mail Classification

Sending huge number of unwanted mails causes security threat to users. In spite of various security approaches, spammers cause much vulnerability in the internet. This paper discusses the efficient methods of using some of the popular algorithms for building a machine learning model which can classify whether a mail is a spam or ham. UCI Machine Learning Repository Spambase Data Set is used for the experiment. The performance of five important machine learning classification algorithms viz. Logistic Regression, Decision Tree, Naive Bayes, KNN and SVM are evaluated in order to train and build an effective machine learning model for email spam detection. Weka tool is used for training and testing the data set.

Performance Evaluation of Machine Learning Algorithms for Email Spam Detection

The traditional anti-spam techniques like Black and White List is not up to the mark in current scenario. The goal of Spam Classification is to distinguish between spam and legitimate mail message. But with the popularization of the Internet, it is challenging to develop spam filters that can effectively eliminate the increasing volumes of unwanted mails automatically before they enter a user's mailbox. Many researchers have been trying to separate spam from legitimate emails using machine learning algorithms based on statistical learning methods. In this paper, we evaluate the performance of Non Linear SVM based classifiers with various kernel functions over Enron Dataset.

/pdf/spam-filtering-using-support-vector-machine-54k3zfa7hm.pdf

Priyanka Chhabra

Papers

Spam Filtering using Support Vector Machine