With exponential growth in the size of computer networks and developed applications, the significant increasing of the potential damage that can be caused by launching attacks is becoming obvious. Meanwhile, Intrusion Detection Systems (IDSs) and Intrusion Prevention Systems (IPSs) are one of the most important defense tools against the sophisticated and ever-growing network attacks. Due to the lack of adequate dataset, anomaly-based approaches in intrusion detection systems are suffering from accurate deployment, analysis and evaluation. There exist a number of such datasets such as DARPA98, KDD99, ISC2012, and ADFA13 that have been used by the researchers to evaluate the performance of their proposed intrusion detection and intrusion prevention approaches. Based on our study over eleven available datasets since 1998, many such datasets are out of date and unreliable to use. Some of these datasets suffer from lack of traffic diversity and volumes, some of them do not cover the variety of attacks, while others anonymized packet information and payload which cannot reflect the current trends, or they lack feature set and metadata. This paper produces a reliable dataset that contains benign and seven common attack network flows, which meets real world criteria and is publicly avaliable. Consequently, the paper evaluates the performance of a comprehensive set of network traffic features and machine learning algorithms to indicate the best set of features for detecting the certain attack categories.

Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization

Review: Intrusion detection system: A comprehensive review

A survey of network anomaly detection techniques

Since the early 1980s, short-term traffic forecasting has been an integral part of most Intelligent Transportation Systems (ITS) research and applications; most effort has gone into developing methodologies that can be used to model traffic characteristics and produce anticipated traffic conditions. Existing literature is voluminous, and has largely used single point data from motorways and has employed univariate mathematical models to predict traffic volumes or travel times. Recent developments in technology and the widespread use of powerful computers and mathematical models allow researchers an unprecedented opportunity to expand horizons and direct work in 10 challenging, yet relatively under researched, directions. It is these existing challenges that we review in this paper and offer suggestions for future work.

Short-term traffic forecasting: Where we are and where we’re going

Machine learning techniques are being widely used to develop an intrusion detection system (IDS) for detecting and classifying cyberattacks at the network-level and the host-level in a timely and automatic manner. However, many challenges arise since malicious attacks are continually changing and are occurring in very large volumes requiring a scalable solution. There are different malware datasets available publicly for further research by cyber security community. However, no existing study has shown the detailed analysis of the performance of various machine learning algorithms on various publicly available datasets. Due to the dynamic nature of malware with continuously changing attacking methods, the malware datasets available publicly are to be updated systematically and benchmarked. In this paper, a deep neural network (DNN), a type of deep learning model, is explored to develop a flexible and effective IDS to detect and classify unforeseen and unpredictable cyberattacks. The continuous change in network behavior and rapid evolution of attacks makes it necessary to evaluate various datasets which are generated over the years through static and dynamic approaches. This type of study facilitates to identify the best algorithm which can effectively work in detecting future cyberattacks. A comprehensive evaluation of experiments of DNNs and other classical machine learning classifiers are shown on various publicly available benchmark malware datasets. The optimal network parameters and network topologies for DNNs are chosen through the following hyperparameter selection methods with KDDCup 99 dataset. All the experiments of DNNs are run till 1,000 epochs with the learning rate varying in the range [0.01-0.5]. The DNN model which performed well on KDDCup 99 is applied on other datasets, such as NSL-KDD, UNSW-NB15, Kyoto, WSN-DS, and CICIDS 2017, to conduct the benchmark. Our DNN model learns the abstract and high-dimensional feature representation of the IDS data by passing them into many hidden layers. Through a rigorous experimental testing, it is confirmed that DNNs perform well in comparison with the classical machine learning classifiers. Finally, we propose a highly scalable and hybrid DNNs framework called scale-hybrid-IDS-AlertNet which can be used in real-time to effectively monitor the network traffic and host-level events to proactively alert possible cyberattacks.

/pdf/deep-learning-approach-for-intelligent-intrusion-detection-19gksnoeyl.pdf

Deep Learning Approach for Intelligent Intrusion Detection System

Anomaly detection in wireless sensor networks: A survey

Cyber-physical systems (CPSs) integrate the computation with physical processes. Embedded computers and networks monitor and control the physical processes, usually with feedback loops where physical processes affect computations and vice versa. CPS was identified as one of the eight research priority areas in the August 2007 report of the President's Council of Advisors on Science and Technology, as CPS will be the core component of many critical infrastructures and industrial control systems in the near future. However, a variety of random failures and cyber attacks exist in CPS, which greatly restrict their growth. Fortunately, an intrusion detection mechanism could take effect for protecting CPS. When a misbehavior is found by the intrusion detector, the appropriate action can be taken immediately so that any harm to the system will be minimized. As CPSs are yet to be defined universally, the application of the instruction detection mechanism remain open presently. As a result, the effort will be made to discuss how to appropriately apply the intrusion detection mechanism to CPS in this paper. By examining the unique properties of CPS, it intends to define the specific requirements first. Then, the design outline of the intrusion detection mechanism in CPS is introduced in terms of the layers of system and specific detection techniques. Finally, some significant research problems are identified for enlightening the subsequent studies.

Intrusion Detection in Cyber-Physical Systems: Techniques and Challenges

Online anomaly detection (AD) is an important technique for monitoring wireless sensor networks (WSNs), which protects WSNs from cyberattacks and random faults. As a scalable and parameter-free unsupervised AD technique, k-nearest neighbor (kNN) algorithm has attracted a lot of attention for its applications in computer networks and WSNs. However, the nature of lazy-learning makes the kNN-based AD schemes difficult to be used in an online manner, especially when communication cost is constrained. In this paper, a new kNN-based AD scheme based on hypergrid intuition is proposed for WSN applications to overcome the lazy-learning problem. Through redefining anomaly from a hypersphere detection region (DR) to a hypercube DR, the computational complexity is reduced significantly. At the same time, an attached coefficient is used to convert a hypergrid structure into a positive coordinate space in order to retain the redundancy for online update and tailor for bit operation. In addition, distributed computing is taken into account, and position of the hypercube is encoded by a few bits only using the bit operation. As a result, the new scheme is able to work successfully in any environment without human interventions. Finally, the experiments with a real WSN data set demonstrate that the proposed scheme is effective and robust.

Scalable Hypergrid k-NN-Based Online Anomaly Detection in Wireless Sensor Networks

Host-based intrusion detection systems (HIDSs), especially anomaly-based, have received much attention over the past few decades. Over time, however, the existing data sets used for evaluation of a HIDS have lost most of their relevance due to the substantial development of computer systems. To fill this gap, ADFA Linux data set (ADFA-LD) is recently released, which is composed of thousands of system call traces collected from a contemporary Linux local server and expects to be a new benchmark for evaluating a HIDS. In this paper, we perform a preliminary analysis of ADFA-LD, in an attempt to extract useful information for developing new host-based anomaly detection systems (HADSs). In accordance with the general concerns arising from the community, some typical features are analysed particularly against ADFA-LD, such as length, common pattern and frequency. Furthermore, we implement a simple k nearest neighbour (kNN)-based HADS to be evaluated using ADFA-LD. The experimental results show that, although an acceptable performance can be acquired for a few types of attack, there is still a long way to fully understand the complex behaviour resulting from a modern computer system and, finally, realise more intelligent HADSs.

Evaluating host-based anomaly detection systems: A preliminary analysis of ADFA-LD

ADFA-LD is a recently released data set for evaluating host-based anomaly detection systems, aiming to substitute the existing benchmark data sets which have failed to reflect the characteristics of modern computer systems. In a previous work, we had attempted to evaluate ADFA-LD with a highly efficient frequency model but the performance is inferior. In this paper, we focus on the other typical technical category that detects anomalies with a short sequence model. In collaboration with the one-class SVM algorithm, a novel anomaly detection system is proposed for ADFA-LD. The numerical experiments demonstrate that it can not only achieve a satisfactory performance, but also reduce the computational cost largely.

Miao Xie

Papers

Anomaly detection in wireless sensor networks: A survey

Intrusion Detection in Cyber-Physical Systems: Techniques and Challenges

Scalable Hypergrid k-NN-Based Online Anomaly Detection in Wireless Sensor Networks

Evaluating host-based anomaly detection systems: A preliminary analysis of ADFA-LD

Evaluating host-based anomaly detection systems: Application of the one-class SVM algorithm to ADFA-LD