Showing papers on "Intrusion detection system published in 2009"

Review: Intrusion detection by machine learning: A review

Abstract: The popularity of using Internet contains some risks of network attacks. Intrusion detection is one major research problem in network security, whose aim is to identify unusual access or attacks to secure internal networks. In literature, intrusion detection systems have been approached by various machine learning techniques. However, there is no a review paper to examine and understand the current status of using machine learning techniques to solve the intrusion detection problems. This chapter reviews 55 related studies in the period between 2000 and 2007 focusing on developing single, hybrid, and ensemble classifiers. Related studies are compared by their classifier design, datasets used, and other experimental setups. Current achievements and limitations in developing intrusion detection systems by machine learning are present and discussed. A number of future research directions are also provided.

ANTIDOTE: understanding and defending against poisoning of anomaly detectors

Abstract: Statistical machine learning techniques have recently garnered increased popularity as a means to improve network design and security. For intrusion detection, such methods build a model for normal behavior from training data and detect attacks as deviations from that model. This process invites adversaries to manipulate the training data so that the learned model fails to detect subsequent attacks.We evaluate poisoning techniques and develop a defense, in the context of a particular anomaly detector - namely the PCA-subspace method for detecting anomalies in backbone networks. For three poisoning schemes, we show how attackers can substantially increase their chance of successfully evading detection by only adding moderate amounts of poisoned data. Moreover such poisoning throws off the balance between false positives and false negatives thereby dramatically reducing the efficacy of the detector.To combat these poisoning activities, we propose an antidote based on techniques from robust statistics and present a new robust PCA-based detector. Poisoning has little effect on the robust model, whereas it significantly distorts the model produced by the original PCA method. Our technique substantially reduces the effectiveness of poisoning for a variety of scenarios and indeed maintains a significantly better balance between false positives and false negatives than the original method when under attack.

Nuzzer: A Large-Scale Device-Free Passive Localization System for Wireless Environments

Abstract: The widespread usage of wireless local area networks and mobile devices has fostered the interest in localization systems for wireless environments. The majority of research in the context of wireless-based localization systems has focused on device-based active localization, in which a device is attached to tracked entities. Recently, device-free passive localization (DfP) has been proposed where the tracked entity is neither required to carry devices nor participate actively in the localization process. DfP systems are based on the fact that RF signals are affected by the presence of people and objects in the environment. The DfP concept enables a wide range of applications including intrusion detection and tracking, border protection, and smart buildings automation. Previous studies have focused on small areas with direct line of sight and/or controlled environments. In this paper, we present the design, implementation and analysis of Nuzzer, a large-scale device-free passive localization system for real environments. Without any additional hardware, it makes use of the already installed wireless data networks to monitor and process changes in the received signal strength (RSS) transmitted from access points at one or more monitoring points. We present probabilistic techniques for DfP localization and evaluate their performance in a typical office building, rich in multipath, with an area of 1500 square meters. Our results show that the Nuzzer system gives device-free location estimates with less than 2 meters median distance error using only two monitoring laptops and three access points. This indicates the suitability of Nuzzer to a large number of application domains.

Intrusion Detection in the Cloud

Abstract: Intrusion Detection Systems (IDS) have been used widely to detect malicious behaviors in network communication and hosts. IDS management is an important capability for distributed IDS solutions, which makes it possible to integrate and handle different types of sensors or collect and synthesize alerts generated from multiple hosts located in the distributed environment. Facing new application scenarios in Cloud Computing, the IDS approaches yield several problems since the operator of the IDS should be the user, not the administrator of the Cloud infrastructure. Extensibility, efficient management, and compatibility to virtualization-based context need to be introduced into many existing IDS implementations.Additionally, the Cloud providers need to enable possibilities to deploy and configure IDS for the user. Within this paper, we summarize several requirements for deploying IDS in the Cloud and propose an extensible IDS architecture for being easily used in a distributed cloud infrastructure.

Intrusion detection using fuzzy association rules

Abstract: Vulnerabilities in common security components such as firewalls are inevitable. Intrusion Detection Systems (IDS) are used as another wall to protect computer systems and to identify corresponding vulnerabilities. In this paper, a novel framework based on data mining techniques is proposed for designing an IDS. In this framework, the classification engine, which is actually the core of the IDS, uses Association Based Classification (ABC). The proposed classification algorithm uses fuzzy association rules for building classifiers. Particularly, the fuzzy association rulesets are exploited as descriptive models of different classes. The compatibility of any new sample (which is to be classified) with different class rulesets is assessed by the use of some matching measures and the class corresponding to the best matched ruleset is declared as the label of the sample. A new method is also proposed to speed up the rule induction algorithm via reducing items that may be included in extracted rules. KDD-99 dataset is used to evaluate the proposed framework. Although results on unseen attacks are not so promising, total detection rate and detection rate of known attacks is significant while false positive rate is kept low. Results are compared with some recent works in the literature using the same dataset. Generally, the proposed approach outperforms other methods, specially in terms of false positive rate.

Neural Network based Intrusion Detection System for critical infrastructures

Abstract: Resiliency and security in control systems such as SCADA and Nuclear plant's in today's world of hackers and malware are a relevant concern. Computer systems used within critical infrastructures to control physical functions are not immune to the threat of cyber attacks and may be potentially vulnerable. Tailoring an intrusion detection system to the specifics of critical infrastructures can significantly improve the security of such systems. The IDS-NNM - Intrusion Detection System using Neural Network based Modeling, is presented in this paper. The main contributions of this work are: 1) the use and analyses of real network data (data recorded from an existing critical infrastructure); 2) the development of a specific window based feature extraction technique; 3) the construction of training dataset using randomly generated intrusion vectors; 4) the use of a combination of two neural network learning algorithms - the Error-Back Propagation and Levenberg-Marquardt, for normal behavior modeling. The presented algorithm was evaluated on previously unseen network data. The IDS-NNM algorithm proved to be capable of capturing all intrusion attempts presented in the network communication while not generating any false alerts.

Network anomaly detection based on wavelet analysis

Abstract: Signal processing techniques have been applied recently for analyzing and detecting network anomalies due to their potential to find novel or unknown intrusions. In this paper, we propose a new network signal modelling technique for detecting network anomalies, combining the wavelet approximation and system identification theory. In order to characterize network traffic behaviors, we present fifteen features and use them as the input signals in our system. We then evaluate our approach with the 1999 DARPA intrusion detection dataset and conduct a comprehensive analysis of the intrusions in the dataset. Evaluation results show that the approach achieves high-detection rates in terms of both attack instances and attack types. Furthermore, we conduct a full day's evaluation in a real large-scale WiFi ISP network where five attack types are successfully detected from over 30 millions flows.

Flow processing and the rise of commodity network hardware

Abstract: The Internet has seen a proliferation of specialized middlebox devices that carry out crucial network functionality such as load balancing, packet inspection and intrusion detection. Recent advances in CPU power, memory, buses and network connectivity have turned commodity PC hardware into a powerful network platform. Furthermore, commodity switch technologies have recently emerged offering the possibility to control the switching of flows in a fine-grained manner. Exploiting these new technologies, we present a new class of network architectures which enables flow processing and forwarding at unprecedented flexibility and low cost.

GT: picking up the truth from the ground for internet traffic

Abstract: Much of Internet traffic modeling, firewall, and intrusion detection research requires traces where some ground truth regarding application and protocol is associated with each packet or flow. This paper presents the design, development and experimental evaluation of gt, an open source software toolset for associating ground truth information with Internet traffic traces. By probing the monitored host's kernel to obtain information on active Internet sessions, gt gathers ground truth at the application level. Preliminary experimental results show that gt's effectiveness comes at little cost in terms of overhead on the hosting machines. Furthermore, when coupled with other packet inspection mechanisms, gt can derive ground truth not only in terms of applications (e.g., e-mail), but also in terms of protocols (e.g., SMTP vs. POP3).

Automatically generating models for botnet detection

Abstract: A botnet is a network of compromised hosts that is under the control of a single, malicious entity, often called the botmaster. We present a system that aims to detect bots, independent of any prior information about the command and control channels or propagation vectors, and without requiring multiple infections for correlation. Our system relies on detection models that target the characteristic fact that every bot receives commands from the botmaster to which it responds in a specific way. These detection models are generated automatically from network traffic traces recorded from actual bot instances. We have implemented the proposed approach and demonstrate that it can extract effective detection models for a variety of different bot families. These models are precise in describing the activity of bots and raise very few false positives.

Packet Classification Algorithms: From Theory to Practice

Abstract: During the past decade, the packet classification problem has been widely studied to accelerate network applications such as access control, traffic engineering and intrusion detection. In our research, we found that although a great number of packet classification algorithms have been proposed in recent years, unfortunately most of them stagnate in mathematical analysis or software simulation stages and few of them have been implemented in commercial products as a generic solution. To fill the gap between theory and practice, in this paper, we propose a novel packet classification algorithm named HyperSplit. Compared to the well-known HiCuts and HSM algorithms, HyperSplit achieves superior performance in terms of classification speed, memory usage and preprocessing time. The practicability of the proposed algorithm is manifested by two facts in our test: HyperSplit is the only algorithm that can successfully handle all the rule sets; HyperSplit is also the only algorithm that reaches more than 6Gbps throughput on the Octeon3860 multi-core platform when tested with 64-byte Ethernet packets against 10K ACL rules.

VMM-based intrusion detection system

Abstract: An intrusion detection system collects architectural level events from a Virtual Machine Monitor where the collected events represent operation of a corresponding Virtual Machine. The events are consolidated into features that are compared with features from a known normal operating system. If an amount of any differences between the collected features and the normal features exceeds a threshold value, a compromised Virtual Machine may be indicated. The comparison thresholds are determined by training on normal and abnormal systems and analyzing the collected events with machine learning algorithms to arrive at a model of normal operation.

Smart cevices for smart environments: Device-free passive detection in real environments

Abstract: Device-free Passive (DfP) localization is a system envisioned to detect, track, and identify entities that do not carry any device, nor participate actively in the localization process. A DfP system allows using nominal WiFi equipment for intrusion detection, without using any extra hardware, adding smartness to any WiFi-enabled device. In this paper, we focus on the detection function of the DfP system in a real environment. We show that the performance of our previously developed algorithms for detection in a controlled environments, which achieved 100% recall and precision, degrades significantly when tested in a real environment. We present an alternative algorithm, based on the maximum likelihood estimator (MLE), that has a significant performance increase in a real environment. Our results show that the recall of the system increases by more than 10% when using the proposed MLE without affecting the system's precision.

A simple and efficient hidden Markov model scheme for host-based anomaly intrusion detection

Abstract: Extensive research activities have been observed on network-based intrusion detection systems (IDSs). However, there are always some attacks that penetrate traffic-profiling-based network IDSs. These attacks often cause very serious damages such as modifying host critical files. A host-based anomaly IDS is an effective complement to the network IDS in addressing this issue. This article proposes a simple data preprocessing approach to speed up a hidden Markov model (HMM) training for system-call-based anomaly intrusion detection. Experiments based on a public database demonstrate that this data preprocessing approach can reduce training time by up to 50 percent with unnoticeable intrusion detection performance degradation, compared to a conventional batch HMM training scheme. More than 58 percent data reduction has been observed compared to our prior incremental HMM training scheme. Although this maximum gain incurs more degradation of false alarm rate performance, the resulting performance is still reasonable.

RRE: A game-theoretic intrusion Response and Recovery Engine

Abstract: Preserving the availability and integrity of networked computing systems in the face of fast-spreading intrusions requires advances not only in detection algorithms, but also in automated response techniques. In this paper, we propose a new approach to automated response called the Response and Recovery Engine (RRE). Our engine employs a game-theoretic response strategy against adversaries modeled as opponents in a two-player Stackelberg stochastic game. RRE applies attack-response trees to analyze undesired security events and their countermeasures using Boolean logic to combine lower-level attack consequences. In addition, RRE accounts for uncertainties in intrusion detection alert notifications. RRE then chooses optimal response actions by solving a partially observable competitive Markov decision process that is automatically derived from attack-response trees. Experimental results show that RRE, using Snort's alerts, can protect large networks for which attack-response trees have more than 900 nodes.

Using static analysis for Ajax intrusion detection

Abstract: We present a static control-flow analysis for JavaScript programs running in a web browser. Our analysis tackles numerous challenges posed by modern web applications including asynchronous communication, frameworks, and dynamic code generation. We use our analysis to extract a model of expected client behavior as seen from the server, and build an intrusion-prevention proxy for the server: the proxy intercepts client requests and disables those that do not meet the expected behavior. We insert random asynchronous requests to foil mimicry attacks. Finally, we evaluate our technique against several real applications and show that it protects against an attack in a widely-used web application.

Orchestra: intrusion detection using parallel execution and monitoring of program variants in user-space

Abstract: In a Multi-Variant Execution Environment (MVEE), several slightly different versions of the same program are executed in lockstep. While this is done, a monitor compares the behavior of the versions at certain synchronization points with the aim of detecting discrepancies which may indicate attacks.As we show, the monitor can be implemented entirely in user space, eliminating the need for kernel modifications. As a result, the monitor is not a part of the trusted code base.We have built a fully functioning MVEE, named Orchestra, and evaluated its effectiveness. We obtained benchmark results on a quad-core system, using two variants which grow the stack in opposite directions. The results show that the overall penalty of simultaneous execution and monitoring of two variants on a multi-core system averages about 15% relative to unprotected conventional execution

A Labeled Data Set for Flow-Based Intrusion Detection

Abstract: Flow-based intrusion detection has recently become a promising security mechanism in high speed networks (1-10 Gbps). Despite the richness in contributions in this field, benchmarking of flow-based IDS is still an open issue. In this paper, we propose the first publicly available, labeled data set for flow-based intrusion detection. The data set aims to be realistic , i.e., representative of real traffic and complete from a labeling perspective. Our goal is to provide such enriched data set for tuning, training and evaluating ID systems. Our setup is based on a honeypot running widely deployed services and directly connected to the Internet, ensuring attack-exposure. The final data set consists of 14.2M flows and more than 98% of them has been labeled.

Application of wavelet-based RF fingerprinting to enhance wireless network security

Abstract: This work continues a trend of developments aimed at exploiting the physical layer of the open systems interconnection (OSI) model to enhance wireless network security. The goal is to augment activity occurring across other OSI layers and provide improved safeguards against unauthorized access. Relative to intrusion detection and anti-spoofing, this paper provides details for a proof-of-concept investigation involving “air monitor” applications where physical equipment constraints are not overly restrictive. In this case, RF fingerprinting is emerging as a viable security measure for providing device-specific identification (manufacturer, model, and/or serial number). RF fingerprint features can be extracted from various regions of collected bursts, the detection of which has been extensively researched. Given reliable burst detection, the near-term challenge is to find robust fingerprint features to improve device distinguishability. This is addressed here using wavelet domain (WD) RF fingerprinting based on dual-tree complex wavelet transform (DT-CWT) features extracted from the non-transient preamble response of OFDM-based 802.11a signals. Intra-manufacturer classification performance is evaluated using four like-model Cisco devices with dissimilar serial numbers. WD fingerprinting effectiveness is demonstrated using Fisher-based multiple discriminant analysis (MDA) with maximum likelihood (ML) classification. The effects of varying channel SNR, burst detection error and dissimilar SNRs for MDA/ML training and classification are considered. Relative to time domain (TD) RF fingerprinting, WD fingerprinting with DT-CWT features emerged as the superior alternative for all scenarios at SNRs below 20 dB while achieving performance gains of up to 8 dB at 80% classification accuracy.

A Game Theoretical Framework on Intrusion Detection in Heterogeneous Networks

Abstract: Due to the dynamic, distributed, and heterogeneous nature of today's networks, intrusion detection systems (IDSs) have become a necessary addition to the security infrastructure and are widely deployed as a complementary line of defense to classical security approaches. In this paper, we address the intrusion detection problem in heterogeneous networks consisting of nodes with different noncorrelated security assets. In our study, two crucial questions are: What are the expected behaviors of rational attackers? What is the optimal strategy of the defenders (IDSs)? We answer the questions by formulating the network intrusion detection as a noncooperative game and performing an in-depth analysis on the Nash equilibrium and the engineering implications behind. Based on our game theoretical analysis, we derive the expected behaviors of rational attackers, the minimum monitor resource requirement, and the optimal strategy of the defenders. We then provide guidelines for IDS design and deployment. We also show how our game theoretical framework can be applied to configure the intrusion detection strategies in realistic scenarios via a case study. Finally, we evaluate the proposed game theoretical framework via simulations. The simulation results show both the correctness of the analytical results and the effectiveness of the proposed guidelines.

Regular Expression Matching on Graphics Hardware for Intrusion Detection

Abstract: The expressive power of regular expressions has been often exploited in network intrusion detection systems, virus scanners, and spam filtering applications. However, the flexible pattern matching functionality of regular expressions in these systems comes with significant overheads in terms of both memory and CPU cycles, since every byte of the inspected input needs to be processed and compared against a large set of regular expressions. In this paper we present the design, implementation and evaluation of a regular expression matching engine running on graphics processing units (GPUs). The significant spare computational power and data parallelism capabilities of modern GPUs permits the efficient matching of multiple inputs at the same time against a large set of regular expressions. Our evaluation shows that regular expression matching on graphics hardware can result to a 48 times speedup over traditional CPU implementations and up to 16 Gbit/s in processing throughput. We demonstrate the feasibility of GPU regular expression matching by implementing it in the popular Snort intrusion detection system, which results to a 60% increase in the packet processing throughput.

An end to the middle

Abstract: The last fifteen years has seen a vast proliferation of middleboxes to solve all manner of persistent limitations in the Internet protocol suite. Examples include firewalls, NATs, load balancers, traffic shapers, deep packet intrusion detection, virtual private networks, network monitors, transparent web caches, content delivery networks, and the list goes on and on. However, most smaller networks in homes, small businesses and the developing world are left without this level of support. Further, the management burden and limitations of middleboxes are apparent even in enterprise networks. We argue for a shift from using proprietary middle-box harware as the dominant tool for managing networks toward using open software running on end hosts. We show that functionality that seemingly must be in the network, such as NATs and traffic prioritization, can be more cheaply, flexibly, and securely provided by distributed software running on end hosts, working in concert with vastly simplified physical network hardware.

Using Rough Set and Support Vector Machine for Network Intrusion Detection System

Abstract: The main function of IDS (Intrusion Detection System) is to protect the system, analyze and predict the behaviors of users. Then these behaviors will be considered an attack or a normal behavior. Though IDS has been developed for many years, the large number of return alert messages makes managers maintain system inefficiently. In this paper, we use RST (Rough Set Theory) and SVM (Support Vector Machine) to detect intrusions. First, RST is used to preprocess the data and reduce the dimensions. Next, the features selected by RST will be sent to SVM model to learn and test respectively. The method is effective to decrease the space density of data. The experiments will compare the results with different methods and show RST and SVM schema could improve the false positive rate and accuracy.

Data mining-based intrusion detectors

Abstract: With popularization of internet, internet attack cases are increasing, and attack methods differs each day, thus information safety problem has became a significant issue all over the world. Nowadays, it is an urgent need to detect, identify and hold up such attacks effectively. The research intends to compare efficiency of machine learning methods in intrusion detection system, including classification tree and support vector machine, with the hope of providing reference for establishing intrusion detection system in future. Compared with other related works in data mining-based intrusion detectors, we proposed to calculate the mean value via sampling different ratios of normal data for each measurement, which lead us to reach a better accuracy rate for observation data in real world. We compared the accuracy, detection rate, false alarm rate for four attack types. More over, it shows better performance than KDD Winner, especially for U2R type and R2L type attacks.

Monitoring smartphones for anomaly detection

Abstract: In this paper we demonstrate how to monitor a smartphone running Symbian operating system and Windows Mobile in order to extract features for anomaly detection. These features are sent to a remote server because running a complex intrusion detection system on this kind of mobile device still is not feasible due to capability and hardware limitations. We give examples on how to compute relevant features and introduce the top ten applications used by mobile phone users based on a study in 2005. The usage of these applications is recorded by a monitoring client and visualized. Additionally, monitoring results of public and self-written malwares are shown. For improving monitoring client performance, Principal Component Analysis was applied which lead to a decrease of about 80 of the amount of monitored features.

Evaluating GPUs for network packet signature matching

Abstract: Modern network devices employ deep packet inspection to enable sophisticated services such as intrusion detection, traffic shaping, and load balancing. At the heart of such services is a signature matching engine that must match packet payloads to multiple signatures at line rates. However, the recent transition to complex regular-expression based signatures coupled with ever-increasing network speeds has rapidly increased the performance requirements of signature matching. Solutions to meet these requirements range from hardwarecentric ASIC/FPGA implementations to software implementations using high-performance microprocessors. In this paper, we propose a programmable signature matching system prototyped on an Nvidia G80 GPU. We first present a detailed architectural and microarchitectural analysis, showing that signature matching is well suited for SIMD processing because of regular control flow and parallelism available at the packet level. Next, we examine two approaches for matching signatures: standard deterministic finite automata (DFAs) and extended finite automata (XFAs), which use far less memory than DFAs but require specialized auxiliary memory and small amounts of computation in most states. We implement a fully functional prototype on the SIMD-based G80 GPU. This system out-performs a Pentium4 by up to 9X and a Niagara-based 32-threaded system by up to 2.3X and shows that GPUs are a promising candidate for signature matching.

RAINBOW: A Robust And Invisible Non-Blind Watermark for Network Flows.

Abstract: Linking network flows is an important problem in intrusion detection as well as anonymity Passive traffic analysis can link flows but requires long periods of observation to reduce errors Watermarking techniques allow for better precision and blind detection, but they do so by introducing significant delays to the traffic flow, enabling attacks that detect and remove the mark, while at the same time slowing down legitimate traffic We propose a new, non-blind watermarking scheme called RAINBOW that is able to use delays hundreds of times smaller than existing watermarks by eliminating the interference caused by the flow in the blind case As a result, our watermark is invisible to detection, as confirmed by experiments using information-theoretic detection tools We analyze the error rates of our scheme based on a mathematical model of network traffic and jitter We also validate the analysis using an implementation running on PlanetLab We find that our scheme generates orders of magnitudes lower rates of false errors than passive traffic analysis, while using only a few hundred observed packets We also extend our scheme so that it is robust to packet drops and repacketization and show that flows can still be reliably linked, though at the cost of somewhat longer observation periods