Packet classification is an enabling function for a variety of Internet applications including quality of service, security, monitoring, and multimedia communications. In order to classify a packet as belonging to a particular flow or set of flows, network nodes must perform a search over a set of filters using multiple fields of the packet as the search key. In general, there have been two major threads of research addressing packet classification, algorithmic and architectural. A few pioneering groups of researchers posed the problem, provided complexity bounds, and offered a collection of algorithmic solutions. Subsequently, the design space has been vigorously explored by many offering new algorithms and improvements on existing algorithms. Given the inability of early algorithms to meet performance constraints imposed by high speed links, researchers in industry and academia devised architectural solutions to the problem. This thread of research produced the most widely-used packet classification device technology, Ternary Content Addressable Memory (TCAM). New architectural research combines intelligent algorithms and novel architectures to eliminate many of the unfavorable characteristics of current TCAMs. We observe that the community appears to be converging on a combined algorithmic and architectural approach to the problem. Using a taxonomy based on the high-level approach to the problem and a minimal set of running examples, we provide a survey of the seminal and recent solutions to the problem. It is our hope to foster a deeper understanding of the various packet classification techniques while providing a useful framework for discerning relationships and distinctions.

/pdf/survey-and-taxonomy-of-packet-classification-techniques-545k9nk529.pdf

Survey and taxonomy of packet classification techniques

In the same way businesses use big data to pursue profits, governments use it to promote the public good.

Big-data applications in the government sector

Packet classification is an enabling technology for next generation network services and often a performance bottleneck in high-performance routers. The performance and capacity of many classification algorithms and devices, including TCAMs, depend upon properties of filter sets and query patterns. Despite the pressing need, no standard performance evaluation tools or filter sets are publicly available. In response to this problem, we present ClassBench, a suite of tools for benchmarking packet classification algorithms and devices. ClassBench includes a filter set generator that produces synthetic filter sets that accurately model the characteristics of real filter sets. Along with varying the size of the filter sets, we provide high-level control over the composition of the filters in the resulting filter set. The tool suite also includes a trace generator that produces a sequence of packet headers to exercise packet classification algorithms with respect to a given filter set. Along with specifying the relative size of the trace, we provide a simple mechanism for controlling locality of reference. While we have already found ClassBench to be very useful in our own research, we seek to eliminate the significant access barriers to realistic test vectors for researchers and initiate a broader discussion to guide the refinement of the tools and codification of a formal benchmarking methodology. (The ClassBench tools are publicly available at the following site: http://www.arl.wustl.edu/~det3/ClassBench/.)

/pdf/classbench-a-packet-classification-benchmark-1serj3rqr6.pdf

ClassBench: a packet classification benchmark

In the structure of ANFIS, there are two different parameter groups: premise and consequence. Training ANFIS means determination of these parameters using an optimization algorithm. In the first ANFIS model developed by Jang, a hybrid learning approach was proposed for training. In this approach, while premise parameters are determined by using gradient descent (GD), consequence parameters are found out with least squares estimation (LSE) method. Since ANFIS has been developed, it is used in modelling and identification of numerous systems and successful results have been achieved. The selection of optimization method utilized in training is very important to get effective results with ANFIS. It is seen that derivate based (GD, LSE etc.) and non-derivative based (heuristic algorithms such us GA, PSO, ABC etc.) algorithms are used in ANFIS training. Nevertheless, it has been observed that there is a trend toward heuristic based ANFIS training algorithms for better performance recently. At the same time, it seems to be proposed in derivative and heuristic based hybrid algorithms. Within the scope of this study, the heuristic and hybrid approaches utilized in ANFIS training are examined in order to guide researchers in their study. In addition, the final status in ANFIS training is evaluated and it is aimed to shed light on further studies related to ANFIS training.

Adaptive network based fuzzy inference system (ANFIS) training approaches: a comprehensive survey

In this paper we introduce a deep neural network based malware detection system that Invincea has developed, which achieves a usable detection rate at an extremely low false positive rate and scales to real world training example volumes on commodity hardware. We show that our system achieves a 95% detection rate at 0.1% false positive rate (FPR), based on more than 400,000 software binaries sourced directly from our customers and internal malware databases. In addition, we describe a non-parametric method for adjusting the classifier’s scores to better represent expected precision in the deployment environment. Our results demonstrate that it is now feasible to quickly train and deploy a low resource, highly accurate machine learning classification model, with false positive rates that approach traditional labor intensive expert rule based malware detection, while also detecting previously unseen malware missed by these traditional approaches. Since machine learning models tend to improve with larger datasizes, we foresee deep neural network classification models gaining in importance as part of a layered network defense strategy in coming years.

Deep neural network based malware detection using two dimensional binary program features

Emerging Internet applications create the need for advanced packet classifiers. We propose a novel multifield classification scheme, called P/sup 2/C, which exploits the strengths of state-of-the-art memory technologies to provide wire-speed classification performance for OC-192 and beyond, in combination with very high storage efficiency and the support of fast incremental updates. Key features of the new scheme are its ability to adapt to the complexity of a classification rule set, whereas the storage requirements and update dynamics can be tuned at the granularity of individual rules. This makes P/sup 2/C suitable for a broad spectrum of applications.

Fast and scalable packet classification

A packet-switched system architecture based on the combination of a single-chip output-buffered switch element and input queues that sort arriving packets on a per-output-port basis is proposed. Scheduling is performed in a distributed two-stage approach. Independent arbiters at each of the inputs resolve input contention. Whereas the output-buffered switch element resolves output contention. As a result of this distribution of functionality, complexity of the input arbiters is only linearly proportional to the number of output ports N, thus offering better scalability than purely input-buffered approaches that require complex centralized schedulers. Since the input queues are used as the main buffering mechanism, only a relatively small amount of memory (on the order of N/sup 2/ packet locations) is required in the shared-memory switch, allowing high-throughput implementations. We present simulation results to demonstrate the high performance and robustness under bursty traffic achieved with the proposed system architecture. A practical implementation in the form of the PRIZMA family of switch chips is outlined, with emphasis on its versatility in scaling in terms of both port speed and number of ports, and its support for quality-of-service mechanisms.

A combined input and output queued packet switched system based on PRIZMA switch on a chip technology

In today's commercial data centers, the computation density grows continuously as the number of hardware components and workloads in units of virtual machines increase. The service availability guaranteed by data centers heavily depends on the reliability of the physical and virtual servers. In this study, we conduct an analysis on 10K virtual and physical machines hosted on five commercial data centers over an observation period of one year. Our objective is to establish a sound understanding of the differences and similarities between failures of physical and virtual machines. We first capture their failure patterns, i.e., the failure rates, the distributions of times between failures and of repair times, as well as, the time and space dependency of failures. Moreover, we correlate failures with the resource capacity and run-time usage to identify the characteristics of failing servers. Finally, we discuss how virtual machine management actions, i.e., consolidation and on/off frequency, impact virtual machine failures.

Failure Analysis of Virtual and Physical Machines: Patterns, Causes and Characteristics

Review of advances in neural networks

For a system in which data packets are to be handled according to one of several rules, depending on two (or more) criteria present in each packet, such as source and destination addresses, a classification method is disclosed that allows to determine the applicable rule by a longest-matching-prefix search operation. Range tokens of non-uniform length are assigned to basic ranges of criterion values so that each combination of input values from a packet can be represented by a particular variable length combination of range tokens. A search tree containing stored rule identifiers is so designed that each particular range token combination, used as input for a longest-matching-prefix lookup operation, will provide the required identifier. Different range token combinations having the same prefix can use the same path to one stored rule identifier, so that this method reduces the storage and time requirements for the classification procedure and allows simple updating when rules change.

Ton Engbersen

Papers

Fast and scalable packet classification

A combined input and output queued packet switched system based on PRIZMA switch on a chip technology

Failure Analysis of Virtual and Physical Machines: Patterns, Causes and Characteristics

Review of advances in neural networks

Method and means for classifying data packets