Complete formal verification is the only known way to guarantee that a system is free of programming errorsWe present our experience in performing the formal, machine-checked verification of the seL4 microkernel from an abstract specification down to its C implementation We assume correctness of compiler, assembly code, and hardware, and we used a unique design approach that fuses formal and operating systems techniques To our knowledge, this is the first formal proof of functional correctness of a complete, general-purpose operating-system kernel Functional correctness means here that the implementation always strictly follows our high-level abstract specification of kernel behaviour This encompasses traditional design and implementation safety properties such as the kernel will never crash, and it will never perform an unsafe operation It also proves much more: we can predict precisely how the kernel will behave in every possible situationseL4, a third-generation microkernel of L4 provenance, comprises 8,700 lines of C code and 600 lines of assembler Its performance is comparable to other high-performance L4 kernels

/pdf/sel4-formal-verification-of-an-os-kernel-20d8n4rs7p.pdf

seL4: formal verification of an OS kernel

In network intrusion detection research, one popular strategy for finding attacks is monitoring a network's activity for anomalies: deviations from profiles of normality previously learned from benign traffic, typically identified using tools borrowed from the machine learning community However, despite extensive academic research one finds a striking gap in terms of actual deployments of such systems: compared with other intrusion detection approaches, machine learning is rarely employed in operational "real world" settings We examine the differences between the network intrusion detection problem and other areas where machine learning regularly finds much more success Our main claim is that the task of finding attacks is fundamentally different from these other applications, making it significantly harder for the intrusion detection community to employ machine learning effectively We support this claim by identifying challenges particular to network intrusion detection, and provide a set of guidelines meant to strengthen future research on anomaly detection

/pdf/outside-the-closed-world-on-using-machine-learning-for-4oj3n5uk26.pdf

Outside the Closed World: On Using Machine Learning for Network Intrusion Detection

We present a payload-based anomaly detector, we call PAYL, for intrusion detection. PAYL models the normal application payload of network traffic in a fully automatic, unsupervised and very effecient fashion. We first compute during a training phase a profile byte frequency distribution and their standard deviation of the application payload flowing to a single host and port. We then use Mahalanobis distance during the detection phase to calculate the similarity of new data against the pre-computed profile. The detector compares this measure against a threshold and generates an alert when the distance of the new input exceeds this threshold. We demonstrate the surprising effectiveness of the method on the 1999 DARPA IDS dataset and a live dataset we collected on the Columbia CS department network. In once case nearly 100% accuracy is achieved with 0.1% false positive rate for port 80 traffic.

/pdf/anomalous-payload-based-network-intrusion-detection-olwhsg8m8h.pdf

Anomalous Payload-Based Network Intrusion Detection

Unlike signature or misuse based intrusion detection techniques, anomaly detection is capable of detecting novel attacks. However, the use of anomaly detection in practice is hampered by a high rate of false alarms. Specification-based techniques have been shown to produce a low rate of false alarms, but are not as effective as anomaly detection in detecting novel attacks, especially when it comes to network probing and denial-of-service attacks. This paper presents a new approach that combines specification-based and anomaly-based intrusion detection, mitigating the weaknesses of the two approaches while magnifying their strengths. Our approach begins with state-machine specifications of network protocols, and augments these state machines with information about statistics that need to be maintained to detect anomalies. We present a specification language in which all of this information can be captured in a succinct manner. We demonstrate the effectiveness of the approach on the 1999 Lincoln Labs intrusion detection evaluation data, where we are able to detect all of the probing and denial-of-service attacks with a low rate of false alarms (less than 10 per day). Whereas feature selection was a crucial step that required a great deal of expertise and insight in the case of previous anomaly detection approaches, we show that the use of protocol specifications in our approach simplifies this problem. Moreover, the machine learning component of our approach is robust enough to operate without human supervision, and fast enough that no sampling techniques need to be employed. As further evidence of effectiveness, we present results of applying our approach to detect stealthy email viruses in an intranet environment.

http://ants.iis.sinica.edu.tw/3BkMJ9lTeWXTSrrvNoKNFDxRm3zFwRR/17/p265-sekar.pdf

Specification-based anomaly detection: a new approach for detecting network intrusions

31 Introduction 42 What is Malware? 4 2.1 Who are the Users and Creators of Malware? . . . . . . . . . . . . . . . 6 3 The Malware Detector 64 Malware Detection Techniques 7 4.1 Anomaly-based Detection . . . . . . . . . . . . . . . . . . . . . . . . . . 94.1.1 Dynamic Anomaly-based Detection . . . . . . . . . . . . . . . . 104.1.2 Static Anomaly-based Detection . . . . . . . . . . . . . . . . . . 154.1.3 Hybrid Anomaly-based Detection . . . . . . . . . . . . . . . . . . 164.2 Speciﬁcation-based Detection . . . . . . . . . . . . . . . . . . . . . . . . 184.2.1 Dynamic Speciﬁcation-based Detection . . . . . . . . . . . . . . 184.2.2 Static Speciﬁcation-based Detection . . . . . . . . . . . . . . . . 264.2.3 Hybrid Speciﬁcation-based Detection . . . . . . . . . . . . . . . 284.3 Signature-based detection . . . . . . . . . . . . . . . . . . . . . . . . . . 314.3.1 Dynamic Signature-based Detection . . . . . . . . . . . . . . . . 334.3.2 Static Signature-based Detection . . . . . . . . . . . . . . . . . . 344.3.3 Hybrid Signature-based Detection . . . . . . . . . . . . . . . . . 38

A Survey of Malware Detection Techniques

High-assurance systems require a level of rigor, in both design and analysis, not typical of conventional systems. This paper provides an overview of the Multiple Independent Levels of Security and Safety (MILS) approach to high-assurance system design for security and safety critical embedded systems. MILS enables the development of a system using manageable units, each of which can be analysed separately, avoiding costly analysis required of more conventional designs. MILS is particularly well suited to embedded systems that must provide guaranteed safety or security properties.

The MILS architecture for high-assurance embedded systems

In 1987, Dorothy Denning published the seminal paper on anomaly detection as applied to intrusion detection on a single system. Her paper sparked a new paradigm in intrusion detection research with the notion that malicious behavior could be distinguished from normal system use. Since that time, a great deal of anomaly detection research based on Denning's original premise has occurred. However, Denning's assumptions about anomalies that originate on a single host have been applied essentially unaltered to networks. In this paper we question the application of Denning's work to network based anomaly detection, along with other assumptions commonly made in network-based detection research. We examine the assumptions underlying selected studies of network anomaly detection and discuss these assumptions in the context of the results from studies of network traffic patterns. The purpose of questioning the old paradigm of anomaly detection as a strategy for network intrusion detection is to reconfirm the paradigm as sound or begin the process of replacing it with a new paradigm in light of changes in the operating environment.

/pdf/challenging-the-anomaly-detection-paradigm-a-provocative-2q507sg5sb.pdf

Challenging the anomaly detection paradigm: a provocative discussion

A new approach to network intrusion detection is needed to solve the monitoring problems of high volume network data and the time constraints for Intrusion Detection System (IDS) management. Most current network IDS's have not been specifically designed for high speed traffic or low maintenance. We propose a solution to these problems which we call NATE, Network Analysis of Anomalous Traffic Events. Our approach features minimal network traffic measurement, an anomaly-based detection method, and a limited attack scope. NATE is similar to other lightweight approaches in its simplified design, but our approach, being anomaly based, should be more efficient in both operation and maintenance than other lightweight approaches. We present the method and perform an empirical test using MIT Lincoln Lab's data.

/pdf/nate-network-analysis-of-anomalous-traffic-events-a-low-cost-1th26rqssu.pdf

NATE: Network Analysis of Anomalous Traffic Events, a low-cost approach

NATE: Network Analysis ofAnomalousTrafficEvents, a low-cost approach

Past efforts at designing and implementing ultra high assurance systems for government security and safety have centered on the concept of a monolithic security kernel responsible for a system-wide security policy. This approach leads to inflexible, overly complex operating systems that are too large to evaluate at the highest assurance levels (e.g., Common Criteria EAL 5 and above). We describe a new multi-layered approach to the design and verification of embedded trustworthy systems that is currently being used in the implementation of real time, embedded applications. The framework supports multiple levels of safety and multiple levels of security, based on the principle of creating separate layers of responsibility and control, with each layer responsible for enforcing its own security policy.

/pdf/a-multi-layered-approach-to-security-in-high-assurance-13i8lg3r56.pdf

Carol Taylor

Papers

The MILS architecture for high-assurance embedded systems

Challenging the anomaly detection paradigm: a provocative discussion

NATE: Network Analysis of Anomalous Traffic Events, a low-cost approach

NATE: Network Analysis ofAnomalousTrafficEvents, a low-cost approach

A multi-layered approach to security in high assurance systems