With the development of the Internet, cyber-attacks are changing rapidly and the cyber security situation is not optimistic. This survey report describes key literature surveys on machine learning (ML) and deep learning (DL) methods for network analysis of intrusion detection and provides a brief tutorial description of each ML/DL method. Papers representing each method were indexed, read, and summarized based on their temporal or thermal correlations. Because data are so important in ML/DL methods, we describe some of the commonly used network datasets used in ML/DL, discuss the challenges of using ML/DL for cybersecurity and provide suggestions for research directions.

Machine Learning and Deep Learning Methods for Cybersecurity

In a computing context, cybersecurity is undergoing massive shifts in technology and its operations in recent days, and data science is driving the change. Extracting security incident patterns or insights from cybersecurity data and building corresponding data-driven model, is the key to make a security system automated and intelligent. To understand and analyze the actual phenomena with data, various scientific methods, machine learning techniques, processes, and systems are used, which is commonly known as data science. In this paper, we focus and briefly discuss on cybersecurity data science, where the data is being gathered from relevant cybersecurity sources, and the analytics complement the latest data-driven patterns for providing more effective security solutions. The concept of cybersecurity data science allows making the computing process more actionable and intelligent as compared to traditional ones in the domain of cybersecurity. We then discuss and summarize a number of associated research issues and future directions. Furthermore, we provide a machine learning based multi-layered framework for the purpose of cybersecurity modeling. Overall, our goal is not only to discuss cybersecurity data science and relevant methods but also to focus the applicability towards data-driven intelligent decision making for protecting the systems from cyber-attacks.

/pdf/cybersecurity-data-science-an-overview-from-machine-learning-2g5zdq7ti3.pdf

Cybersecurity data science: an overview from machine learning perspective

Intrusion detection can identify unknown attacks from network traffics and has been an effective means of network security. Nowadays, existing methods for network anomaly detection are usually based on traditional machine learning models, such as KNN, SVM, etc. Although these methods can obtain some outstanding features, they get a relatively low accuracy and rely heavily on manual design of traffic features, which has been obsolete in the age of big data. To solve the problems of low accuracy and feature engineering in intrusion detection, a traffic anomaly detection model BAT is proposed. The BAT model combines BLSTM (Bidirectional Long Short-term memory) and attention mechanism. Attention mechanism is used to screen the network flow vector composed of packet vectors generated by the BLSTM model, which can obtain the key features for network traffic classification. In addition, we adopt multiple convolutional layers to capture the local features of traffic data. As multiple convolutional layers are used to process data samples, we refer BAT model as BAT-MC. The softmax classifier is used for network traffic classification. The proposed end-to-end model does not use any feature engineering skills and can automatically learn the key features of the hierarchy. It can well describe the network traffic behavior and improve the ability of anomaly detection effectively. We test our model on a public benchmark dataset, and the experimental results demonstrate our model has better performance than other comparison methods.

/pdf/bat-deep-learning-methods-on-network-intrusion-detection-51nukofjb5.pdf

BAT: Deep Learning Methods on Network Intrusion Detection Using NSL-KDD Dataset

Cyber security has recently received enormous attention in today’s security concerns, due to the popularity of the Internet-of-Things (IoT), the tremendous growth of computer networks, and the huge number of relevant applications. Thus, detecting various cyber-attacks or anomalies in a network and building an effective intrusion detection system that performs an essential role in today’s security is becoming more important. Artificial intelligence, particularly machine learning techniques, can be used for building such a data-driven intelligent intrusion detection system. In order to achieve this goal, in this paper, we present an Intrusion Detection Tree (“IntruDTree”) machine-learning-based security model that first takes into account the ranking of security features according to their importance and then build a tree-based generalized intrusion detection model based on the selected important features. This model is not only effective in terms of prediction accuracy for unseen test cases but also minimizes the computational complexity of the model by reducing the feature dimensions. Finally, the effectiveness of our IntruDTree model was examined by conducting experiments on cybersecurity datasets and computing the precision, recall, fscore, accuracy, and ROC values to evaluate. We also compare the outcome results of IntruDTree model with several traditional popular machine learning methods such as the naive Bayes classifier, logistic regression, support vector machines, and k-nearest neighbor, to analyze the effectiveness of the resulting security model.

IntruDTree: A Machine Learning Based Cyber Security Intrusion Detection Model

In imbalanced network traffic, malicious cyber-attacks can often hide in large amounts of normal data. It exhibits a high degree of stealth and obfuscation in cyberspace, making it difficult for Network Intrusion Detection System(NIDS) to ensure the accuracy and timeliness of detection. This paper researches machine learning and deep learning for intrusion detection in imbalanced network traffic. It proposes a novel Difficult Set Sampling Technique(DSSTE) algorithm to tackle the class imbalance problem. First, use the Edited Nearest Neighbor(ENN) algorithm to divide the imbalanced training set into the difficult set and the easy set. Next, use the KMeans algorithm to compress the majority samples in the difficult set to reduce the majority. Zoom in and out the minority samples’ continuous attributes in the difficult set synthesize new samples to increase the minority number. Finally, the easy set, the compressed set of majority in the difficult, and the minority in the difficult set are combined with its augmentation samples to make up a new training set. The algorithm reduces the imbalance of the original training set and provides targeted data augment for the minority class that needs to learn. It enables the classifier to learn the differences in the training stage better and improve classification performance. To verify the proposed method, we conduct experiments on the classic intrusion dataset NSL-KDD and the newer and comprehensive intrusion dataset CSE-CIC-IDS2018. We use classical classification models: random forest(RF), Support Vector Machine(SVM), XGBoost, Long and Short-term Memory(LSTM), AlexNet, Mini-VGGNet. We compare the other 24 methods; the experimental results demonstrate that our proposed DSSTE algorithm outperforms the other methods.

/pdf/intrusion-detection-of-imbalanced-network-traffic-based-on-3rx48wmmmc.pdf

Intrusion Detection of Imbalanced Network Traffic Based on Machine Learning and Deep Learning

These days, with the tremendous growth of network-based service and shared information on networks, the risk of network attacks and intrusions increases too, therefore network security and protecting the network is getting more significance than before. Intrusion Detection System (IDS) is one of the solutions to detect attacks and anomalies in the network. The ever rising new intrusion or attack types causes difficulties for their detection, therefore Data mining techniques has been widely applied in network intrusion detection systems for extracting useful knowledge from large number of network data to detect intrusions. Many clustering and classification algorithms are used in IDS, therefore improving the functionality of these algorithms will improve IDS performance. This paper focuses on improving KNN classifier in existing intrusion detection task which combines K-MEANS clustering and KNN classification.

/pdf/intrusion-detection-using-a-novel-hybrid-method-4nucj7j1zl.pdf

Intrusion Detection using a Novel Hybrid Method Incorporating an Improved KNN

In order to make computer systems completely secure, in addition to firewalls and other intrusion protection devices, other systems called intrusion detection systems (IDS) are needed to detect intrusion and provide solutions to counter the intruder if he penetrated through firewall, antivirus and other security devices. Many IDS have been developed based on machine learning techniques. Specifically, advanced detection approaches created by combining or integrating multiple learning techniques have shown better detection performance than general single learning techniques. This paper proposes an improvement for a feature representation approach, namely the cluster center and nearest neighbor (CANN) approach.

/pdf/a-novel-cluster-based-intrusion-detection-approach-tra4w8vgnm.pdf

A Novel Cluster-based Intrusion Detection Approach Integrating Multiple Learning Techniques

/pdf/an-improved-intrusion-detection-system-based-on-combined-5fkl9gcdj2.pdf

Hossein Shapoorifard

Papers

Intrusion Detection using a Novel Hybrid Method Incorporating an Improved KNN

A Novel Cluster-based Intrusion Detection Approach Integrating Multiple Learning Techniques

An Improved Intrusion Detection System based on Combined Cluster Centers and Nearest Neighbors