In network intrusion detection research, one popular strategy for finding attacks is monitoring a network's activity for anomalies: deviations from profiles of normality previously learned from benign traffic, typically identified using tools borrowed from the machine learning community However, despite extensive academic research one finds a striking gap in terms of actual deployments of such systems: compared with other intrusion detection approaches, machine learning is rarely employed in operational "real world" settings We examine the differences between the network intrusion detection problem and other areas where machine learning regularly finds much more success Our main claim is that the task of finding attacks is fundamentally different from these other applications, making it significantly harder for the intrusion detection community to employ machine learning effectively We support this claim by identifying challenges particular to network intrusion detection, and provide a set of guidelines meant to strengthen future research on anomaly detection

/pdf/outside-the-closed-world-on-using-machine-learning-for-4oj3n5uk26.pdf

Outside the Closed World: On Using Machine Learning for Network Intrusion Detection

We present a new kind of network perimeter monitoring strategy, which focuses on recognizing the infection and coordination dialog that occurs during a successful malware infection BotHunter is an application designed to track the two-way communication flows between internal assets and external entities, developing an evidence trail of data exchanges that match a state-based infection sequence model BotHunter consists of a correlation engine that is driven by three malware-focused network packet sensors, each charged with detecting specific stages of the malware infection process, including inbound scanning, exploit usage, egg downloading, outbound bot coordination dialog, and outbound attack propagation The BotHunter correlator then ties together the dialog trail of inbound intrusion alarms with those outbound communication patterns that are highly indicative of successful local host infection When a sequence of evidence is found to match BotHunter's infection dialog model, a consolidated report is produced to capture all the relevant events and event sources that played a role during the infection process We refer to this analytical strategy of matching the dialog flows between internal assets and the broader Internet as dialog-based correlation, and contrast this strategy to other intrusion detection and alert correlation methods We present our experimental results using BotHunter in both virtual and live testing environments, and discuss our Internet release of the BotHunter prototype BotHunter is made available both for operational use and to help stimulate research in understanding the life cycle of malware infections

/pdf/bothunter-detecting-malware-infection-through-ids-driven-3wf3mqkin8.pdf

BotHunter: detecting malware infection through IDS-driven dialog correlation

Nowadays, numerous journals and conferences have published articles related to context-aware systems, indicating many researchers' interest. Therefore, the goal of this paper is to review the works that were published in journals, suggest a new classification framework of context-aware systems, and explore each feature of classification framework. This paper is based on a literature review of context-aware systems from 2000 to 2007 using a keyword index and article title search. The classification framework is developed based on the architecture of context-aware systems, which consists of the following five layers: concept and research layer, network layer, middleware layer, application layer and user infrastructure layer. The articles are categorized based on the classification framework. This paper allows researchers to extract several lessons learned that are important for the implementation of context-aware systems.

Context-aware systems

An alarming trend in malware attacks is that they are armed with stealthy techniques to detect, evade, and subvert malware detection facilities of the victim. On the defensive side, a fundamental limitation of traditional host-based anti-malware systems is that they run inside the very hosts they are protecting ("in the box"), making them vulnerable to counter-detection and subversion by malware. To address this limitation, recent solutions based on virtual machine (VM) technologies advocate placing the malware detection facilities outside of the protected VM ("out of the box"). However, they gain tamper resistance at the cost of losing the native, semantic view of the host which is enjoyed by the "in the box" approach, thus leading to a technical challenge known as the semantic gap.In this paper, we present the design, implementation, and evaluation of VMwatcher - an "out-of-the-box" approach that overcomes the semantic gap challenge. A new technique called guest view casting is developed to systematically reconstruct internal semantic views (e.g., files, processes, and kernel modules) of a VM from the outside in a non-intrusive manner. Specifically, the new technique casts semantic definitions of guest OS data structures and functions on virtual machine monitor (VMM)-level VM states, so that the semantic view can be reconstructed. With the semantic gap bridged, we identify two unique malware detection capabilities: (1) view comparison-based malware detection and its demonstration in rootkit detection and (2) "out-of-the-box" deployment of host-based anti-malware software with improved detection accuracy and tamper-resistance. We have implemented a proof-of-concept prototype on both Linux and Windows platforms and our experimental results with real-world malware, including elusive kernel-level rootkits, demonstrate its practicality and effectiveness.

/pdf/stealthy-malware-detection-through-vmm-based-out-of-the-box-joxd2myaqk.pdf

Stealthy malware detection through vmm-based "out-of-the-box" semantic view reconstruction

As the web continues to play an ever increasing role in information exchange, so too is it becoming the prevailing platform for infecting vulnerable hosts. In this paper, we provide a detailed study of the pervasiveness of so-called drive-by downloads on the Internet. Drive-by downloads are caused by URLs that attempt to exploit their visitors and cause malware to be installed and run automatically. Over a period of 10 months we processed billions of URLs, and our results shows that a non-trivial amount, of over 3 million malicious URLs, initiate drive-by downloads. An even more troubling finding is that approximately 1.3% of the incoming search queries to Google's search engine returned at least one URL labeled as malicious in the results page. We also explore several aspects of the drive-by downloads problem. Specifically, we study the relationship between the user browsing habits and exposure to malware, the techniques used to lure the user into the malware distribution networks, and the different properties of these networks.

/pdf/all-your-iframes-point-to-us-4i8f6zsmmu.pdf

All your iFRAMEs point to Us

Methods, systems, and media for detecting covert malware are provided. In accordance with some embodiments, a method for detecting covert malware in a computing environment is provided, the method comprising: generating simulated user activity outside of the computing environment; conveying the simulated user activity to an application inside the computing environment; and determining whether a decoy corresponding to the simulated user activity has been accessed by an unauthorized entity.

Methods, systems, and media for detecting covert malware

In accordance with some embodiments of the present invention, systems and methods that protect an application from attacks are provided. In some embodiments of the present invention, traffic from a communication network is received by an anomaly detection component. The anomaly detection component monitors the received traffic and routes the traffic either to the protected application or to a honeypot, where the honeypot shares all state information with the application. If the received traffic is routed to the honeypot, the honeypot monitors the traffic for an attack. If an attack occurs, the honeypot repairs the protected application (e.g., discarding any state changes incurred from the attack, reverting to previously saved state information, etc.).

Systems and methods for detecting and inhibiting attacks using honeypots

We present Shadow Honeypots, a novel hybrid architecture that combines the best features of honeypots and anomaly detection. At a high level, we use a variety of anomaly detectors to monitor all traffic to a protected network/service. Traffic that is considered anomalous is processed by a "shadow honeypot" to determine the accuracy of the anomaly prediction. The shadow is an instance of the protected software that shares all internal state with a regular ("production") instance of the application, and is instrumented to detect potential attacks. Attacks against the shadow are caught, and any incurred state changes are discarded. Legitimate traffic that was misclassified will be validated by the shadow and will be handled correctly by the system transparently to the end user. The outcome of processing a request by the shadow is used to filter future attack instances and could be used to update the anomaly detector.

Our architecture allows system designers to fine-tune systems for performance, since false positives will be filtered by the shadow. Contrary to regular honeypots, our architecture can be used both for server and client applications. We demonstrate the feasibility of our approach in a proof-of-concept implementation of the Shadow Honeypot architecture for the Apache web server and the Mozilla Firefox browser. We show that despite a considerable overhead in the instrumentation of the shadow honeypot (up to 20% for Apache), the overall impact on the system is diminished by the ability to minimize the rate of false-positives.

/pdf/detecting-targeted-attacks-using-shadow-honeypots-1l6b5px8xj.pdf

Detecting targeted attacks using shadow honeypots

In accordance with the present invention, computer implemented methods and systems are provided that allow an application to automatically recover from software failures and attacks Using one or more sensors, failures may be detected in the application In response to detecting the failure, the portion of the application's code that caused the failure is isolated Using the input vectors that caused the failure, information regarding the failure (eg, the type of failure), a core dump file (eg, stack trace), etc, an emulator-based vaccine that repairs the failure is constructed In response to verifying that the vaccine repaired the failure, the application is automatically updated with the emulator-based vaccine without user intervention Application community features that efficiently use the resources available in software monoculture is also provided An application community may be defined that includes a plurality of devices and the application's code maybe divided into smaller portions of code, which are assigned to each of the plurality of devices for monitoring Each device also notifies the other devices of the failure

Methods and systems for repairing applications

In the past decade, there have been numerous efforts in ubiquitous. computing. For home networks, we believe that ubiquitous computing requires a global-scale system that is securable, administered by, multiple independent nonspecialist administrators, and integrates off-the-shelf hardware and software. In this system every home owner acts as an administrator of the network in the home. We are developing such a system based on the Session Initiation Protocol (SIP), with Bluetooth devices for location sensing and the Service Location Protocol (SLP) for service discovery. We also introduce context-aware location information to augment device discovery bind user communication. The system builds on our CINEMA infrastructure.

Stylianos Sidiroglou

Papers

Methods, systems, and media for detecting covert malware

Systems and methods for detecting and inhibiting attacks using honeypots

Detecting targeted attacks using shadow honeypots

Methods and systems for repairing applications

Ubiquitous computing in home networks