Toward developing a systematic approach to generate benchmark datasets for intrusion detection

Software engineers are faced with a dilemma. They want to write general and wellstructured programs that are flexible and easy to maintain. On the other hand, generality has a price: efficiency. A specialized program solving a particular problem is often significantly faster than a general program. However, the development of specialized software is time-consuming, and is likely to exceed the production of today’s programmers. New techniques are required to solve this so-called software crisis. Partial evaluation is a program specialization technique that reconciles the benefits of generality with efficiency. This thesis presents an automatic partial evaluator for the Ansi C programming language. The content of this thesis is analysis and transformation of C programs. We develop several analyses that support the transformation of a program into its generating extension. A generating extension is a program that produces specialized programs when executed on parts of the input. The thesis contains the following main results. • We develop a generating-extension transformation, and describe specialization of the various parts of C, including pointers and structures. • We develop constraint-based inter-procedural pointer and binding-time analysis. Both analyses are specified via non-standard type inference systems, and implemented by constraint solving. • We develop a side-effect and an in-use analysis. These analyses are developed in the classical monotone data-flow analysis framework. Some intriguing similarities with constraint-based analysis are observed. • We investigate separate and incremental program analysis and transformation. Realistic programs are structured into modules, which break down inter-procedural analyses that need global information about functions. • We prove that partial evaluation at most can accomplish linear speedup, and develop an automatic speedup analysis. • We study the stronger transformation technique driving, and initiate the development of generating super-extensions. The developments in this thesis are supported by an implementation. Throughout the chapters we present empirical results.

Program Analysis and Specialization for the C Programming Language

Eight sites participated in the second Defense Advanced Research Projects Agency (DARPA) off-line intrusion detection evaluation in 1999. A test bed generated live background traffic similar to that on a government site containing hundreds of users on thousands of hosts. More than 200 instances of 58 attack types were launched against victim UNIX and Windows NT hosts in three weeks of training data and two weeks of test data. False-alarm rates were low (less than 10 per day). The best detection was provided by network-based systems for old probe and old denial-of-service (DoS) attacks and by host-based systems for Solaris user-to-root (U2R) attacks. The best overall performance would have been provided by a combined system that used both host- and network-based intrusion detection. Detection accuracy was poor for previously unseen, new, stealthy and Windows NT attacks. Ten of the 58 attack types were completely missed by all systems. Systems missed attacks because signatures for old attacks did not generalize to new attacks, auditing was not available on all hosts, and protocols and TCP services were not analyzed at all or to the depth required. Promising capabilities were demonstrated by host-based systems, anomaly detection systems and a system that performs forensic analysis on file system data.

/pdf/the-1999-darpa-off-line-intrusion-detection-evaluation-46mz7k1dy5.pdf

The 1999 DARPA off-line intrusion detection evaluation

We present a new kind of network perimeter monitoring strategy, which focuses on recognizing the infection and coordination dialog that occurs during a successful malware infection BotHunter is an application designed to track the two-way communication flows between internal assets and external entities, developing an evidence trail of data exchanges that match a state-based infection sequence model BotHunter consists of a correlation engine that is driven by three malware-focused network packet sensors, each charged with detecting specific stages of the malware infection process, including inbound scanning, exploit usage, egg downloading, outbound bot coordination dialog, and outbound attack propagation The BotHunter correlator then ties together the dialog trail of inbound intrusion alarms with those outbound communication patterns that are highly indicative of successful local host infection When a sequence of evidence is found to match BotHunter's infection dialog model, a consolidated report is produced to capture all the relevant events and event sources that played a role during the infection process We refer to this analytical strategy of matching the dialog flows between internal assets and the broader Internet as dialog-based correlation, and contrast this strategy to other intrusion detection and alert correlation methods We present our experimental results using BotHunter in both virtual and live testing environments, and discuss our Internet release of the BotHunter prototype BotHunter is made available both for operational use and to help stimulate research in understanding the life cycle of malware infections

/pdf/bothunter-detecting-malware-infection-through-ids-driven-3wf3mqkin8.pdf

BotHunter: detecting malware infection through IDS-driven dialog correlation

The DARPA/MIT Lincoln Laboratory off-line intrusion detection evaluation data set is the most widely used public benchmark for testing intrusion detection systems Our investigation of the 1999 background network traffic suggests the presence of simulation artifacts that would lead to overoptimistic evaluation of network anomaly detection systems The effect can be mitigated without knowledge of specific artifacts by mixing real traffic into the simulation, although the method requires that both the system and the real traffic be analyzed and possibly modified to ensure that the system does not model the simulated traffic independently of the real traffic

/pdf/an-analysis-of-the-1999-darpa-lincoln-laboratory-evaluation-1jsc30gt77.pdf

An analysis of the 1999 DARPA/lincoln Laboratory evaluation data for network anomaly detection

STATL is an extensible state/transition-based attack description language designed to support intrusion detection. The language allows one to describe computer penetrations as sequences of actions that an attacker performs to compromise a computer system. A STATL description of an attack scenario can be used by an intrusion detection system to analyze a stream of events and detect possible ongoing intrusions. Since intrusion detection is performed in different domains (i.e., the network or the hosts) and in different operating environments (e.g., Linux, Solaris, or Windows NT), it is useful to have an extensible language that can be easily tailored to different target environments. STATL defines domain-independent features of attack scenarios and provides constructs for extending the language to describe attacks in particular domains and environments. The STATL language has been successfully used in describing both network-based and host-based attacks, and it has been tailored to very different environments, e.g., Sun Microsystems' Solaris and Microsoft's Windows NT. An implementation of the runtime support for the STATL language has been developed and a toolset of intrusion detection systems based on STATL has been implemented. The toolset was used in a recent intrusion detection evaluation effort, delivering very favorable results. This paper presents the details of the STATL syntax and its semantics. Real examples from both the host and network-based extensions of the language are also presented.

/pdf/statl-an-attack-language-for-state-based-intrusion-detection-27nmtu1xo1.pdf

STATL: an attack language for state-based intrusion detection

Describes a suite of intrusion detection tools developed by the Reliable Software Group at the University of California at Santa Barbara (UCSB). The tool suite is based on the state transition analysis technique (STAT), in which computer penetrations are specified as sequences of actions that cause transitions in the security state of a system. This general approach has been extended and tailored to perform intrusion detection in different domains and environments. The most recent STAT-based intrusion detection systems were developed following a framework-based approach, and the resulting design uses a "core" module that embodies the domain-independent characteristics of the STAT approach. This generic core is extended in a well-defined way to implement intrusion detection systems for different domains and environments. The approach supports software reuse, portability and extendibility, and it allows for the optimization of critical functionalities.

The STAT tool suite

UNISEX is a UNIX-based symbolic executor for Pascal. The UNISEX system provides an environment for both testing and formally verifying Pascal programs. The system supports a large subset of Pascal, runs on UNIX and provides the user with a variety of debugging features to help in the difficult task of program validation. This paper contains a brief introduction to symbolic execution, followed by an overview of the features of UNISEX, a discussion of the UNISEX Pascal language, and some of the implementation details for the UNISEX system. Finally, some of the problems encountered when designing and implementing the system are discussed as well as future directions.

Unisex: A unix‐based symbolic executor for pascal

The number of intrusion detection systems (IDSs) is large and growing Most IDSs are signature based, which means that they include signatures for some collection of known attacks, and monitor an event stream looking for instances of any signature in their collection There is an enormous duplication of effort within the IDS community, as each newly discovered attack requires independent specification for each IDS Sharing of signature collections has obvious advantages for the IDS community as a whole, mainly by (1) allowing better allocation of scarce resources (developers and researchers) and (2) supporting peer review of signature collections, which can lead to better signatures and better detectors Snort is an IDS with a large published collection of signatures This paper considers automated translation of Snort rules to STATL scenarios Automatically translating Snort rules to STATL scenarios has the practical effect of allowing the use of Snort’s large signature collection with NetSTAT sensors, with essentially no new work as new Snort signatures are developed A snort2statl translator has been developed that implements the described translation scheme Most of the signaturespecifying elements of Snort’s rule language are easy to translate to STATL, but developing the translation scheme and its implementation, and then translating the complete set of rules in the standard Snort rule set into STATL scenarios, raised a few issues that are discussed

http://www.raid-symposium.org/raid2001/papers/eckmann_raid2001.pdf

Steven T. Eckmann

Papers

STATL: an attack language for state-based intrusion detection

The STAT tool suite

Unisex: A unix‐based symbolic executor for pascal

Translating Snort rules to STATL scenarios

INATEST: an interactive environment for testing formal specifications