Anomaly detection is an important problem that has been researched within diverse research areas and application domains. Many anomaly detection techniques have been specifically developed for certain application domains, while others are more generic. This survey tries to provide a structured and comprehensive overview of the research on anomaly detection. We have grouped existing techniques into different categories based on the underlying approach adopted by each technique. For each category we have identified key assumptions, which are used by the techniques to differentiate between normal and anomalous behavior. When applying a given technique to a particular domain, these assumptions can be used as guidelines to assess the effectiveness of the technique in that domain. For each category, we provide a basic anomaly detection technique, and then show how the different existing techniques in that category are variants of the basic technique. This template provides an easier and more succinct understanding of the techniques belonging to each category. Further, for each category, we identify the advantages and disadvantages of the techniques in that category. We also provide a discussion on the computational complexity of the techniques since it is an important issue in real application domains. We hope that this survey will provide a better understanding of the different directions in which research has been done on this topic, and how techniques developed in one area can be applied in domains for which they were not intended to begin with.

/pdf/anomaly-detection-a-survey-1x33g9m0a3.pdf

Anomaly detection: A survey

The Smart Grid, regarded as the next generation power grid, uses two-way flows of electricity and information to create a widely distributed automated energy delivery network. In this article, we survey the literature till 2011 on the enabling technologies for the Smart Grid. We explore three major systems, namely the smart infrastructure system, the smart management system, and the smart protection system. We also propose possible future directions in each system. colorred{Specifically, for the smart infrastructure system, we explore the smart energy subsystem, the smart information subsystem, and the smart communication subsystem.} For the smart management system, we explore various management objectives, such as improving energy efficiency, profiling demand, maximizing utility, reducing cost, and controlling emission. We also explore various management methods to achieve these objectives. For the smart protection system, we explore various failure protection mechanisms which improve the reliability of the Smart Grid, and explore the security and privacy issues in the Smart Grid.

/pdf/smart-grid-the-new-and-improved-power-grid-a-survey-4jypbbgv3j.pdf

Smart Grid — The New and Improved Power Grid: A Survey

Smart Grid - The New and Improved Power Grid:

https://www.robots.ox.ac.uk/~davidc/pubs/NDreview2014.pdf

Review: A review of novelty detection

With the increasing advances in hardware technology for data collection, and advances in software technology (databases) for data organization, computer scientists have increasingly participated in the latest advancements of the outlier analysis field. Computer scientists, specifically, approach this field based on their practical experiences in managing large amounts of data, and with far fewer assumptions the data can be of any type, structured or unstructured, and may be extremely large. Outlier Analysisis a comprehensive exposition, as understood by data mining experts, statisticians and computer scientists. The book has been organized carefully, and emphasis was placed on simplifying the content, so that students and practitioners can also benefit. Chapters will typically cover one of three areas: methods and techniques commonly used in outlier analysis, such as linear methods, proximity-based methods, subspace methods, and supervised methods; data domains, such as, text, categorical, mixed-attribute, time-series, streaming, discrete sequence, spatial and network data; and key applications of these methods as applied to diverse domains such as credit card fraud detection, intrusion detection, medical diagnosis, earth science, web log analytics, and social network analysis are covered.

Outlier Analysis

Network anomaly detection is a vibrant research area. Researchers have approached this problem using various techniques such as artificial intelligence, machine learning, and state machine modeling. In this paper, we first review these anomaly detection methods and then describe in detail a statistical signal processing technique based on abrupt change detection. We show that this signal processing technique is effective at detecting several network anomalies. Case studies from real network data that demonstrate the power of the signal processing approach to network anomaly detection are presented. The application of signal processing techniques to this area is still in its infancy, and we believe that it has great potential to enhance the field, and thereby improve the reliability of IP networks.

/pdf/anomaly-detection-in-ip-networks-499hnudue8.pdf

Anomaly detection in IP networks

In third generation (3G) wireless data networks, repeated requests for popular data items can exacerbate the already scarce wireless spectrum. In this paper we propose an architectural and protocol framework that allows 3G service providers to host efficient content distribution services. We offload the spectrum intensive task of content distribution to an ad-hoc network. Less mobile users (resident subscribers) are provided incentives to cache popular data items while mobile users (transit subscribers) access this data from resident subscribers through the ad-hoc network. Since the participants of this data distribution network act as selfish agents, they may collude to maximize their individual payoff. Our proposed protocol discourages potential collusion scenarios. In this architecture the goal (social function) of the 3G service provider is to have the selfishly motivated resident subscribers service as many data requests as possible. However, the choice of which set of items to cache is left to the individual user. The caching activity among the different users can be modeled as a market sharing game. In this work, we study the Nash equilibria of market sharing games and the performance of such equilibria in terms of a social function. These games are a special case of congestion games that have been studied in the economics literature. In particular, pure strategy Nash equilibria for this set of games exist. We give a polynomial-time algorithm to find a pure strategy Nash equilibrium for a special case while it is is NP-Hard to do so in the general case. As for the performance of Nash equilibria, we show that the price of anarchy -- the worst-case ratio between the social function at any Nash equilibrium and at the social optimum -- can be upper bounded by a factor of 2. When the popularity follows a Zipf distribution, the price of anarchy is bounded by 1.45 in the special case where caching any item has a positive reward for all players. We prove that the selfish behavior of computationally bounded agents converges to an approximate Nash equilibrium in a finite number of improvements. Furthermore, we show that even with one improvement by each player, an O(log n) approximate solution can be obtained. Our simulation scenarios show that the price of anarchy is 30% better than that of the worst-case analysis and that the system quickly (1 or 2 steps) converges to a Nash equilibrium.

/pdf/market-sharing-games-applied-to-content-distribution-in-ad-3b4srkq1oo.pdf

Market sharing games applied to content distribution in ad-hoc networks

In recent years the power grid has been undergoing transformative changes due to the greater penetration of renewable energy sources and increased focus on power demand shaping. These innovative transformations on the grid require a flexible IP-based communication grid that is reliable and secure. In this article we describe an IPbased decentralized and data-centric information infrastructure that can reliably, securely, and cost-effectively support the operation and innovative applications of the next generation grid. The proposed infrastructure differs from a typical distributed system since it addresses the specific requirements of power applications such as security, distributed data sources, latency sensitive data transactions and real time event updates. The work presented here paves the way for a future data-centric power network infrastructure.

A secure decentralized data-centric information infrastructure for smart grid

Timely interaction between an SDN controller and switches is crucial to many SDN applications---e.g., fast rerouting during link failure and fine-grained traffic engineering in data centers. However, it is not well understood how the control plane in SDN switches impacts these applications. To this end, we conduct a comprehensive measurement study using four types of production SDN switches. Our measurements show that control actions, such as rule installation, have surprisingly high latency, due to both software implementation inefficiencies and fundamental traits of switch hardware.

/pdf/measuring-control-plane-latency-in-sdn-enabled-switches-493q27rlo9.pdf

Measuring control plane latency in SDN-enabled switches

In this paper we propose cloud-based demand response (CDR), a novel demand response architecture for fast response times in large scale deployments. The proposed architecture is in contrast to master/slave based demand response where the participants directly interact with the utility using host address-centric communication. CDR leverages data-centric communication, publisher/subscriber and topic-based group communication to make demand response secure, scalable and reliable. To the utility, CDR appears to be a black box function call that takes an input from the utility, e.g., power deficit and gives an output to the utility, e.g., power reduction per customer and the corresponding price incentive. Using this implementation framework, we propose two market-based distributed algorithms (bisection and Illinois methods). The proposed algorithms exhibit at least exponentially fast convergence with O(1) iteration as the number of customers grows and outperform prior work of the dual gradient method in terms of convergence speed while keeping the same messaging overhead.

Marina Thottan

Papers

Anomaly detection in IP networks

Market sharing games applied to content distribution in ad-hoc networks

A secure decentralized data-centric information infrastructure for smart grid

Measuring control plane latency in SDN-enabled switches

Cloud-based demand response for smart grid: Architecture and distributed algorithms