Data Mining Practical Machine Learning Tools and Techniques

Distributed Denial of Service attack (DDoS) is recognized to be one of the most catastrophic attacks against various digital communication entities. Software-defined networking (SDN) is an emerging technology for computer networks that uses open protocols for controlling switches and routers placed at the network edges by using specialized open programmable interfaces. In this article, a detailed study on DDoS threats prevalent in SDN is presented. First, SDN features are examined from the perspective of security, and then a discussion on SDN security features is done. Further, two viewpoints on protecting networks against DDoS attacks are presented. In the first view, SDN utilizes its abilities to secure conventional networks. In the second view, SDN may become a victim of the threat itself because of the centralized control mechanism. The main focus of this research work is on discovering critical security implications in SDN while reviewing the current ongoing research studies. By emphasizing the available state-of-the-art techniques, an extensive review of the advancement of SDN security is provided to the research and IT communities.

/pdf/software-defined-networking-based-ddos-defense-mechanisms-1ig7q6orwp.pdf

Software-defined Networking-based DDoS Defense Mechanisms

\n At the advent of advanced wireless technology and contemporary computing paradigms, Distributed Denial of Service (DDoS) attacks on Web-based services have not only increased exponentially in number, but also in the degree of sophistication; hence the need for detecting these attacks within the ocean of communication packets is extremely important. DDoS attacks were initially projected toward the network and transport layers. Over the years, attackers have shifted their offensive strategies toward the application layer. The application layer attacks are potentially more detrimental and stealthier because of the attack traffic and the benign traffic flows being indistinguishable. The distributed nature of these attacks is difficult to combat as they may affect tangible computing resources apart from network bandwidth consumption. In addition, smart devices connected to the Internet can be infected and used as botnets to launch DDoS attacks. In this paper, we propose a novel deep neural network-based detection mechanism that uses feed-forward back-propagation for accurately discovering multiple application layer DDoS attacks. The proposed neural network architecture can identify and use the most relevant high level features of packet flows with an accuracy of 98% on the state-of-the-art dataset containing various forms of DDoS attacks.

DeepDetect: Detection of Distributed Denial of Service Attacks Using Deep Learning

IP Addresses are a central part of packet- and flow-based network data. However, visualization and similarity computation of IP Addresses are challenging to due the missing natural order. This paper presents a novel similarity measure IP2Vec for IP Addresses that builds on ideas from Word2Vec, a popular approach in text mining. The key idea is to learn similarities by extracting available context information from network data. IP Addresses are similar if they appear in similar contexts. Thus, IP2Vec is automatically derived from the given network data set. The proposed approach is evaluated experimentally on two public flow-based data sets. In particular, we demonstrate the effectiveness of clustering IP Addresses within a botnet data set. In addition, we use visualization methods to analyse the learned similarities in more detail. These experiments indicate that IP2Vec is well suited to capture the similarity of IP Addresses based on their network communications.

IP2Vec: Learning Similarities Between IP Addresses

With increment in dependency on web technology, a commensurate increase has been noted in destructive attempts to disrupt the essential web technologies, hence leading to service failures. Web servers that run on Hypertext Transfer Protocol (HTTP) are exposed to denial-of-service (DoS) attacks. A sophisticated version of this attack known as distributed denial of service (DDOS) is among the most dangerous Internet attacks, with the ability to overwhelm a web server, thereby slowing it down and potentially taking it down completely. This paper reviewed 12 recent detection of DDoS attack at the application layer published between January 2014 and December 2018. A summary of each detection method is summarised in table view, along with in-depth critical analysis, for future studies to conduct research pertaining to detection of HTTP DDoS attack.

/pdf/review-of-recent-detection-methods-for-http-ddos-attack-4v1ddrjwiq.pdf

Review of Recent Detection Methods for HTTP DDoS Attack

The SSH Brute force attack is one of the most prevalent attacks in computer networks. These attacks aim to gain ineligible access to users' accounts by trying plenty of different password combinations. The detection of this type of attack at the network level can overcome the scalability issue of host-based detection methods. In this paper, we provide a machine learning approach for the detection of SSH brute force attacks at the network level. Since extracting discriminative features for any machine learning task is a fundamental step, we explain the process of extracting discriminative features for the detection of brute force attacks. We incorporate domain knowledge about SSH brute force attacks as well as the analysis of a representative collection of the data to define the features. We collected real SSH traffic from a campus network. We also generated some failed login data that a legitimate user who has forgotten his/her password can produce as normal traffic that can be similar to the SSH brute force attack traffic. Our inspection on the collected brute force Netflow data and the manually produced SSH failed login data showed that the Netflow features are not discriminative enough to discern brute force traffic from the failed login traffic produced by a legitimate user. We introduced an aggregation of Netflows to extract the proper features for building machine learning models. Our results show that the models built upon these features provide excellent performances for the detection of brute force attacks.

Detection of SSH Brute Force Attacks Using Aggregated Netflow Data

Distributed Denial of Service (DDoS) attacks are a popular and inexpensive form of cyber attacks. Application layer DDoS attacks utilize legitimate application layer requests to overwhelm a web server. These attacks are a major threat to Internet applications and web services. The main goal of these attacks is to make the services unavailable to legitimate users by overwhelming the resources on a web server. They look valid in connection and protocol characteristics, which makes them difficult to detect. In this paper, we propose a detection method for the application layer DDoS attacks, which is based on user behavior anomaly detection. We extract instances of user behaviors requesting resources from HTTP web server logs. We apply the Principle Component Analysis (PCA) subspace anomaly detection method for the detection of anomalous behavior instances. Web server logs from a web server hosting a student resource portal were collected as experimental data. We also generated nine different HTTP DDoS attacks through penetration testing. Our performance results on the collected data show that using PCAsubspace anomaly detection on user behavior data can detect application layer DDoS attacks, even if they are trying to mimic a normal user’s behavior at some level.

User Behavior Anomaly Detection for Application Layer DDoS Attacks

The integrity of modern network communications is constantly being challenged by more sophisticated intrusion techniques. Attackers are consistently shifting to stealthier and more complex forms of attacks in an attempt to bypass known mitigation strategies. In recent years, attackers have begun to focus their attack efforts on the application layer, allowing them to produce attacks that can exploit known issues within specific application protocols. Slow HTTP Denial of Service attacks are one such attack variant, which targets the HTTP protocol and can imitate legitimate user traffic in order to deny resources from a service. Successful mitigation of this attack type requires network analysts to evaluate large quantities of network traffic to identify and block intrusive traffic. The issue, is that the number of legitimate traffic instances can far outnumber the amount of attack instances, making detection problematic. Machine learning techniques can be used to aid in detection, but the large level of imbalance between normal (majority) and attack (minority) instances can lead to inaccurate detection results. In this work, we evaluate the use of data sampling to produce varying class distributions in order to counteract the effects of severely imbalanced Slow HTTP DoS big datasets. We also detail our process for collecting real-world representative Slow HTTP DoS attack traffic from a live network environment to create our datasets. Five class distributions are generated to evaluate the Slow HTTP DoS detection performance of eight machine learning techniques. Our results show that the optimal learner and class distribution combination is that of Random Forest with a 65:35 distribution ratio, obtaining an AUC value of 0.99904. Further, we determine through the use of significance testing, that the use of sampling techniques can significantly increase learner performance when detecting Slow HTTP DoS attack traffic.

/pdf/impact-of-class-distribution-on-the-detection-of-slow-http-2yd7q5kb8g.pdf

Impact of class distribution on the detection of slow HTTP DoS attacks using Big Data

Attackers can leverage several techniques to compromise computer networks, ranging from sophisticated malware to DDoS (Distributed Denial of Service) attacks that target the application layer. Application layer DDoS attacks, such as Slow Read, are implemented with just enough traffic to tie up CPU or memory resources causing web and application servers to go offline. Such attacks can mimic legitimate network requests making them difficult to detect. They also utilize less volume than traditional DDoS attacks. These low volume attack methods can often go undetected by network security solutions until it is too late. In this paper, we explore the use of machine learners for detecting Slow Read DDoS attacks on web servers at the application layer. Our approach uses a generated dataset based upon Netflow data collected at the application layer on a live network environment. Our Netflow data uses the IP Flow Information Export (IPFIX) standard providing significant flexibility and features. These Netflow features can process and handle a growing amount of traffic and have worked well in our previous DDoS work detecting evasion techniques. Our generated dataset consists of real-world network data collected from a production network. We use eight different classifiers to build Slow Read attack detection models. Our wide selection of learners provides us with a more comprehensive analysis of Slow Read detection models. Experimental results show that the machine learners were quite successful in identifying the Slow Read attacks with a high detection and low false alarm rate. The experiment demonstrates that our chosen Netflow features are discriminative enough to detect such attacks accurately

Utilizing Netflow Data to Detect Slow Read Attacks

In this work, we outline a procedure for collecting and labeling Man-in-the-Middle (MITM) attack traffic. Our capture procedure allows for the collection of real-world representative data using a full-scale network environment. MITM attacks are typically performed with the purpose of intercepting information amongst two networked machines. This enables the attacker to gain access to otherwise confidential communications and potentially alter said communications maliciously. MITM attacks are still a very common attack that can be implemented with relative ease across a variety of network environments. Our work establishes experimental procedures for enacting three prevalent MITM attack variants through penetration testing. The process for data collection is defined, along with our approach on gathering real-world, representative data. We also present a novel labeling procedure based on the inherent behaviors of each MITM attack variant. Our work aims to address the challenges associated with collecting such data within a live production environment, as well as identify the impact MITM attacks have on traffic behavior. We also present a case study to provide some quantitative analysis regarding the data collected.

Chad Calvert

Papers

Detection of SSH Brute Force Attacks Using Aggregated Netflow Data

User Behavior Anomaly Detection for Application Layer DDoS Attacks

Impact of class distribution on the detection of slow HTTP DoS attacks using Big Data

Utilizing Netflow Data to Detect Slow Read Attacks

A Procedure for Collecting and Labeling Man-in-the-Middle Attack Traffic