Showing papers on "Traffic classification published in 2013"

Network Traffic Classification Using Correlation Information

Abstract: Traffic classification has wide applications in network management, from security monitoring to quality of service measurements. Recent research tends to apply machine learning techniques to flow statistical feature based classification methods. The nearest neighbor (NN)-based method has exhibited superior classification performance. It also has several important advantages, such as no requirements of training procedure, no risk of overfitting of parameters, and naturally being able to handle a huge number of classes. However, the performance of NN classifier can be severely affected if the size of training data is small. In this paper, we propose a novel nonparametric approach for traffic classification, which can improve the classification performance effectively by incorporating correlated information into the classification process. We analyze the new classification approach and its performance benefit from both theoretical and empirical perspectives. A large number of experiments are carried out on two real-world traffic data sets to validate the proposed approach. The results show the traffic classification performance can be improved significantly even under the extreme difficult circumstance of very few training samples.

NetworkProfiler: Towards automatic fingerprinting of Android apps

Abstract: Network operators need to have a clear visibility into the applications running in their network. This is critical for both security and network management. Recent years have seen an exponential growth in the number of smart phone apps which has complicated this task. Traditional methods of traffic classification are no longer sufficient as the majority of this smart phone app traffic is carried over HTTP/HTTPS. Keeping up with the new applications that come up everyday is very challenging and time-consuming. We present a novel technique for automatically generating network profiles for identifying Android apps in the HTTP traffic. A network profile consists of fingerprints, i.e., unique characteristics of network behavior, that can be used to identify an app. To profile an Android app, we run the app automatically in an emulator and collect the network traces. We have developed a novel UI fuzzing technique for running the app such that different execution paths are exercised, which is necessary to build a comprehensive network profile. We have also developed a light-weight technique, for extracting fingerprints, that is based on identifying invariants in the generated traces. We used our technique to generate network profiles for thousands of apps. Using our network profiles we were able to detect the presence of these apps in real-world network traffic logs from a cellular provider.

Internet Traffic Classification by Aggregating Correlated Naive Bayes Predictions

Abstract: This paper presents a novel traffic classification scheme to improve classification performance when few training data are available. In the proposed scheme, traffic flows are described using the discretized statistical features and flow correlation information is modeled by bag-of-flow (BoF). We solve the BoF-based traffic classification in a classifier combination framework and theoretically analyze the performance benefit. Furthermore, a new BoF-based traffic classification method is proposed to aggregate the naive Bayes (NB) predictions of the correlated flows. We also present an analysis on prediction error sensitivity of the aggregation strategies. Finally, a large number of experiments are carried out on two large-scale real-world traffic datasets to evaluate the proposed scheme. The experimental results show that the proposed scheme can achieve much better classification performance than existing state-of-the-art traffic classification methods.

An Effective Network Traffic Classification Method with Unknown Flow Detection

Abstract: Traffic classification technique is an essential tool for network and system security in the complex environments such as cloud computing based environment. The state-of-the-art traffic classification methods aim to take the advantages of flow statistical features and machine learning techniques, however the classification performance is severely affected by limited supervised information and unknown applications. To achieve effective network traffic classification, we propose a new method to tackle the problem of unknown applications in the crucial situation of a small supervised training set. The proposed method possesses the superior capability of detecting unknown flows generated by unknown applications and utilizing the correlation information among real-world network traffic to boost the classification performance. A theoretical analysis is provided to confirm performance benefit of the proposed method. Moreover, the comprehensive performance evaluation conducted on two real-world network traffic datasets shows that the proposed scheme outperforms the existing methods in the critical network environment.

Reviewing traffic classification

Abstract: Traffic classification has received increasing attention in the last years. It aims at offering the ability to automatically recognize the application that has generated a given stream of packets from the direct and passive observation of the individual packets, or stream of packets, flowing in the network. This ability is instrumental to a number of activities that are of extreme interest to carriers, Internet service providers and network administrators in general. Indeed, traffic classification is the basic block that is required to enable any traffic management operations, from differentiating traffic pricing and treatment (e.g., policing, shaping, etc.), to security operations (e.g., firewalling, filtering, anomaly detection, etc.). Up to few years ago, almost any Internet application was using well-known transport layer protocol ports that easily allowed its identification. More recently, the number of applications using random or non-standard ports has dramatically increased (e.g. Skype, BitTorrent, VPNs, etc.). Moreover, often network applications are configured to use well-known protocol ports assigned to other applications (e.g. TCP port 80 originally reserved for Web traffic) attempting to disguise their presence. For these reasons, and for the importance of correctly classifying traffic flows, novel approaches based respectively on packet inspection, statistical and machine learning techniques, and behavioral methods have been investigated and are becoming standard practice. In this chapter, we discuss the main trend in the field of traffic classification and we describe some of the main proposals of the research community. We complete this chapter by developing two examples of behavioral classifiers: both use supervised machine learning algorithms for classifications, but each is based on different features to describe the traffic. After presenting them, we compare their performance using a large dataset, showing the benefits and drawback of each approach.

Internet Traffic Matrices: A Primer

Abstract: The increasing demand of various services from the Internet has led to an exponential growth of Internet traffic in the last decade, and that growth is likely to continue. With this demand comes the increasing importance of network operations management, planning, provisioning and traffic engineering. A key input into these processes is the traffic matrix, and this is the focus of this chapter. The traffic matrix represents the volumes of traffic from sources to destinations in a network. Here, we first explore the various issues involved in measuring and characterising these matrices. The insights obtained are used to develop models of the traffic, depending on the properties of traffic to be captured: temporal, spatial or spatio-temporal properties. The models are then used in various applications, such as the recovery of traffic matrices, network optimisation and engineering activities, anomaly detection and the synthesis of artificial traffic matrices for testing routing protocols. We conclude the chapter by summarising open questions in Internet traffic matrix research and providing a list resources useful for the researcher and practitioner.

A supervised machine learning approach to classify host roles on line using sFlow

Abstract: Classifying host roles based on network traffic behavior is valuable for network security analysis and detecting security policy violation. Behavior-based network security analysis has advantages over traditional approaches such as code patterns or signatures. Modeling host roles based on network flow data is challenging because of the huge volume of network traffic and overlap among host roles. Many studies of network traffic classification have focused on classifying applications such as web, peer-to-peer, and DNS traffic. In general, machine learning approaches have been applied on classifying applications, security awareness, and anomaly detection. In this paper, we present a supervised machine learning approach that use On-Line Support Vector Machine and Decision Tree to classify host roles. We collect sFlow data from main gateways of a large campus network. We classify different roles, namely, clients versus servers, regular web non-email servers versus web email servers, clients at personal offices versus public places of laboratories and libraries, and personal office clients from two different colleges. We achieved very high classification accuracy, i.e., 99.2% accuracy in classifying clients versus servers, 100% accuracy in classifying regular web non-email servers versus web email servers, 93.3% accuracy in classifying clients at personnel offices versus public places, and 93.3% accuracy in classifying clients at personal offices from two different colleges.

DataTraffic Monitoring and Analysis: from measurement, classification, and anomaly detection to quality of experience

Abstract: High-Performance Network Traffic Processing Systems Using Commodity Hardware.- Active Techniques for Available Bandwidth Estimation: Comparison and Application.- Internet Topology Discovery.- Internet PoP Level Maps.- Analysis of Packet Transmission Processes in Peer-to-Peer Networks by Statistical Inference Methods.- Reviewing Traffic Classification.- A Methodological Overview on Anomaly Detection.- Changepoint Detection Techniques for VoIP Traffic.- Distribution-Based Anomaly Detection in Network Traffic.- From Packets to People: Quality of Experience as a New Measurement Challenge.- Internet Video Delivery in YouTube: From Traffic Measurements to Quality of Experience.- Quality Evaluation in Peer-to-Peer IPTV Services.- Cross-Layer FEC-Based Mechanism for Packet Loss Resilient Video Transmission.- Approaches for Utility-Based QoE-Driven Optimization of Network Resource Allocation for Multimedia Services.- Active Techniques for Available Bandwidth Estimation: Comparison and Application.- Internet Topology Discovery.- Internet PoP Level Maps.- Analysis of Packet Transmission Processes in Peer-to-Peer Networks by Statistical Inference Methods.- Reviewing Traffic Classification.- A Methodological Overview on Anomaly Detection.- Changepoint Detection Techniques for VoIP Traffic.- Distribution-Based Anomaly Detection in Network Traffic.- From Packets to People: Quality of Experience as a New Measurement Challenge.- Internet Video Delivery in YouTube: From Traffic Measurements to Quality of Experience.- Quality Evaluation in Peer-to-Peer IPTV Services.- Cross-Layer FEC-Based Mechanism for Packet Loss Resilient Video Transmission.- Approaches for Utility-Based QoE-Driven Optimization of Network Resource Allocation for Multimedia Services.

ReSurf: Reconstructing web-surfing activity from network traffic

Abstract: More and more applications and services move to the web and this has led to web traffic amounting to as much as 80% of all network traffic. At the same time, most traffic classification efforts stop once they correctly label a flow as web or HTTP. In this paper, we focus on understanding what happens “under the hood” of HTTP traffic. Our first contribution is ReSurf, a systematic approach to reconstruct web-surfing activity starting from raw network data with more than 91% recall and 95% precision over four real network traces. Our second contribution is an extensive analysis of web activity across these traces. By utilizing ReSurf, we study web-surfing behaviors in terms of user requests and transitions between websites (e.g. the click-through history of following hyperlinks). A surprising result is the prevalence of advertising and tracking services that are being accessed during web-surfing that are without the user's explicit consent. In our traces, we found that with 90% chance a user will access such a service after just three user requests (or “clicks”). We believe that our methodology and findings provide valuable insights into modern traffic that can allow: (a) network administrators to better manage and protect their networks, (b) traffic regulators to protect the rights of on-line users, and (c) researchers to better understand the evolution of the traffic from modern websites.

PeerRush: mining for unwanted p2p traffic

Abstract: In this paper we present PeerRush, a novel system for the identification of unwanted P2P traffic. Unlike most previous work, PeerRush goes beyond P2P traffic detection, and can accurately categorize the detected P2P traffic and attribute it to specific P2P applications, including malicious applications such as P2P botnets. PeerRush achieves these results without the need of deep packet inspection, and can accurately identify applications that use encrypted P2P traffic. We implemented a prototype version of PeerRush and performed an extensive evaluation of the system over a variety of P2P traffic datasets. Our results show that we can detect all the considered types of P2P traffic with up to 99.5% true positives and 0.1% false positives. Furthermore, PeerRush can attribute the P2P traffic to a specific P2P application with a misclassification rate of 0.68% or less.

Detection and classification of peer-to-peer traffic: A survey

Abstract: The emergence of new Internet paradigms has changed the common properties of network data, increasing the bandwidth consumption and balancing traffic in both directions. These facts raise important challenges, making it necessary to devise effective solutions for managing network traffic. Since traditional methods are rather ineffective and easily bypassed, particular attention has been paid to the development of new approaches for traffic classification. This article surveys the studies on peer-to-peer traffic detection and classification, making an extended review of the literature. Furthermore, it provides a comprehensive analysis of the concepts and strategies for network monitoring.

Streaming traffic classification method and apparatus

Abstract: In a method of classifying streaming media data, one or more media servers are identified. In response, the media servers are added to a list of media servers. It is determined that one or more messages sent by one or more clients are sent to a first media server included on the list of media servers. In response, the one or more clients are added to a list of clients that is associated with the first media server. It is determined that a request for streaming media is sent to the first media server by a first client included on the list of clients. In response, a rule to determine whether subsequent traffic between the first media server and the first client is utilized. Data packets within the subsequent traffic are classified as high priority when the data packets satisfy the rule.

Towards efficient implementation of packet classifiers in SDN/OpenFlow

Abstract: Traffic classification is a core problem underlying efficient implementation of network services. In this work we draw from our experience in classifier design for commercial systems to address this problem in SDN and OpenFlow. We identify methods from other fields of computer science and show research directions that can be applied for efficient design of packet classifiers. Proposed abstractions and design patterns can significantly reduce requirements on network elements and enable deployment of functionality that would be infeasible in a traditional way.

On the detection of card-sharing traffic through wavelet analysis and Support Vector Machines

Abstract: In the last years, the interest in methods and techniques for circumventing the security of the available digital video broadcasting systems is continuously increasing. Digital TV providers are struggling to restrict access to their video contents only to authorized users, by deploying more and more sophisticated conditional access systems. At the state-of-the-art, the most significant menace is the card-sharing activity which exploits a known weakness allowing an authorized subscriber to provide access to digital contents to a potentially large group of unauthorized ones connected over a communication network. This is usually realized by using ad hoc customized devices. Detecting the presence of these illegal systems on a network, by recognizing their related traffic is an issue of primary importance. Unfortunately, to avoid the identification of such traffic, payload obfuscation strategies based on encryption are often used, hindering packet inspection techniques. This paper presents a strategy for the detection of card-sharing traffic, empowered by machine-learning-driven traffic classification techniques and based on the natural capability of wavelet analysis to decompose a traffic time series into several component series associated with particular time and frequency scales and hence allowing its observation at different frequency component levels and with different resolutions. These ideas have been used for the proof-of-concept implementation of an SVM-based binary classification scheme that relies only on time regularities of the traffic and not on the packet contents and hence is immune to payload obfuscation techniques.

Fine-grained Traffic Classification based on Functional Separation

Abstract: SUMMARY Current efforts to classify Internet traffic highlight accuracy. Previous studies have focused on the detection of major applications such as P2P and streaming applications. However, these applications can generate various types of traffic which are often considered as minor and ignorant traffic portions. As network applications become more complex, the price paid for not concentrating on minor traffic classes is in reduction of accuracy and completeness. In this context, we propose a fine-grained traffic classification scheme and its detailed method, called functional separation. Our proposal can detect, according to functionalities, different types of traffic generated by a single application and should increase completeness by reducing the amount of undetected traffic. We verify our method with real-world traffic. Our performance comparison against existing DPI-based classification frameworks shows that the fine-grained classification scheme achieves consistently higher accuracyand completeness. Copyright © 2013 John Wiley & Sons, Ltd.

High throughput and programmable online trafficclassifier on FPGA

Abstract: Machine learning (ML) algorithms have been shown to be effective in classifying the dynamic internet traffic today. Using additional features and sophisticated ML techniques can improve accuracy and can classify a broad range of application classes. Realizing such classifiers to meet high data rates is challenging. In this paper, we propose two architectures to realize complete online traffic classifier using flow-level features. First, we develop a traffic classifier based on C4.5 decision tree algorithm and Entropy-MDL discretization algorithm. It achieves an accuracy of 97.92% when classifying a traffic trace consisting of eight application classes. Next, we accelerate our classifier using two architectures on FPGA. One architecture stores the classifier in on-chip distributed RAM. It is designed to sustain a high throughput. The other architecture stores the classifier in block RAM. It is designed to operate with small hardware footprint and thus built at low hardware cost. Experimental results show that our high throughput architecture can sustain a throughput of $550$ Gbps assuming 40 Byte packet size. Our low cost architecture demonstrates a 22% better resource efficiency than the high throughput design. It can be easily replicated to achieve $449$ Gbps while supporting 160 input traffic streams concurrently. Both architectures are parameterizable and programmable to support any binary-tree-based traffic classifier. We develop a tool which allows users to easily map a binary-tree-based classifier to hardware. The tool takes a classifier as input and automatically generates the Verilog code for the corresponding hardware architecture.

Resource scheduling in a mobile communication network supporting machine-to-machine (m2m) and user equipment (ue) traffic

Abstract: A system (35) and method (44) that differentiates between Machine-to-Machine (M2M) traffic and User Equipment (UE) traffic when scheduling radio Resource Units (RUs) to M2M terminals (12, 13) and UEs (15, 16) in an Long Term Evolution (LTE) network. Prior to allocation of RUs, the available RUs are divided (48) into two disjoint sets— a UE-specific set (dedicated for UE users) and an M2M-specific set (dedicated for M2M terminals). A hybrid scheduler (70) allocates RUs to M2M terminals from the M2M~$peeifk set only. Any unallocated RUs from the M2M-specifie set and the RUs assigned to the UE-speciflc set are then allocated to the UEs. New M2M-specific Quality of Service Class indicators (QCIs) (60) are defined as well for classifying M2M traffic only. These new QCis are disjoint from the existing QCIs (33), which classify the UE traffic only. The M2M-speciik QCIs separate M2M traffic classification from UE traffic classification so as not to impact UE human users' Quality of Experience (QoE).

Data mining to identify malicious activity

Abstract: Systems and methods may determine suspicious network traffic. A monitoring system comprising a processor in communication with a network may monitor network traffic to or from an asset associated with the network. The monitoring system may assess the network traffic to determine a source and/or destination for the network traffic anchor content of the network traffic. The monitoring system may determine whether the network traffic is suspicious network traffic based on the assessed source and/or destination and/or content. When the network traffic is determined to be suspicious network traffic, the monitoring system may capture metadata associated with the suspicious network traffic and store the metadata in a database in communication with the processor. When the network traffic is not determined to be suspicious network traffic, the monitoring system may disregard metadata associated with the network traffic.

Service level agreement validation via service traffic sample-and-replay

Abstract: In one embodiment, a device samples actual service traffic at a device in a computer network, and generates real-time statistics on distribution of various packet header parameters of the sampled traffic that influence forwarding in the computer network. As such, the device may generate and transmit synthetic measurement traffic according to the distribution. For instance, in one embodiment, the synthetic traffic may be a replay of actual service traffic with an indication that the replayed traffic is synthetic, while in another embodiment, newly generated synthetic measurement traffic may have packet header parameters substantially matching the sampled traffic.

Comparison of Deep Packet Inspection (DPI) Tools for Traffic Classification

Abstract: Nowadays, there are many tools, which are being able to classify the traffic in computer networks. Each of these tools claims to have certain accuracy, but it is a hard task to asses which tool is better, because they are tested on various datasets. Therefore, we made an approach to create a dataset, which can be used to test all the traffic classifiers. In order to do that, we used our system to collect the complete packets from the network interfaces. The packets are grouped into flows, and each flow is collected together with the process name taken from Windows / Linux sockets, so the researchers do not only have the full payloads, but also they are provided the information which application created the flow. Therefore, the dataset is useful for testing Deep Packet Inspection (DPI) tools, as well as statistical, and port-based classifiers. The dataset was created in a fully manual way, which ensures that all the time parameters inside the dataset are comparable with the parameters of the usual network data of the same type. The system for collecting of the data, as well as the dataset, are made available to the public. Afterwards, we compared the accuracy of classification on our dataset of PACE, OpenDPI, NDPI, Libprotoident, NBAR, four different variants of L7-filter, and a statistic-based tool developed at UPC. We performed a comprehensive evaluation of the classifiers on different levels of granularity: application level, content level, and service provider level. We found out that the best performing classifier on our dataset is PACE. From the non-commercial tools, NDPI and Libprotoident provided the most accurate results, while the worst accuracy we obtained from all 4 versions of L7-filter.

SANTaClass: A Self Adaptive Network Traffic Classification system

Abstract: A critical aspect of network management from an operator's perspective is the ability to understand or classify all traffic that traverses the network. The failure of port based traffic classification technique triggered an interest in discovering signatures based on packet content. However, this approach involves manually reverse engineering all the applications/protocols that need to be identified. This suffers from the problem of scalability; keeping up with the new applications that come up everyday is very challenging and time-consuming. Moreover, traditional approach of developing signatures once and using them in different networks suffers from low coverage. In this work, we present a novel fully automated packet payload content (PPC) based network traffic classification system that addresses the above shortcomings. Our system learns new application signatures in the network where classification is desired. Further more, our system adapts the signatures as the traffic for an application changes. Based on real traces from several service providers, we show that our system is capable of detecting (1) tunneled or wrapped applications, (2) applications that use random ports, and (3) new applications. Moreover, it is robust to routing asymmetry, an important requirement in large ISPs, and has a very high (>99.5%) detection rate. Finally, our system is easy to deploy and setup and performs classification in real-time.

Traffic classification: Issues and challenges

Abstract: Traffic classification has been extensively examined in recent years, as it is widely used in network management, design, security, advertising and research. In the past few years, the traffic classification techniques have been evolved along with the development of Internet protocols and applications, and many approaches have been investigated, proposed and developed. Nowadays, the ever increasing network bandwidth, the constantly sophisticated applications and the growth incentives to confuse classification systems to avoid filtering or blocking are among the reasons that traffic classification remains one of the hot areas in network research. In this paper, we first attempt to present an analysis of the existing traffic classification techniques, and dwell on their issues and challenges, then outline some recommendations that can improve the performance of traffic classification systems.

A joint parametric prediction model for wireless internet traffic using Hidden Markov Model

Abstract: Addressing performance related issues of networks and ensuring better Quality of Service (QoS) for end-users calls for simple, tractable and realistic traffic models. The work reported here focuses on modelling the Wireless Internet traffic using realistic traffic traces collected over wireless networks and forecasting the end-to-end QoS parameters for the networks. A measurement framework is set-up to collect the QoS parameters and a traffic model is designed based on Hidden Markov Model considering joint distribution of End to End Delay (E2ED or d), Inter-Packet Delay Variation (IPDV) and Packet Size. States are mapped to the four traffic classes namely conversational, streaming, interactive, and background. The model is validated by forecasting QoS parameters and the results are shown to be within the tolerance limit.

Measuring the accuracy of open-source payload-based traffic classifiers using popular Internet applications

Abstract: Open-source payload-based traffic classifiers are frequently used as a source of ground truth in the traffic classification research field. However, there have been no comprehensive studies that provide evidence that the classifications produced by these software tools are sufficiently accurate for this purpose. In this paper, we present the results of an investigation into the accuracy of four open-source traffic classifiers (L7 Filter, nDPI, libprotoident and tstat) using packet traces captured while using a known selection of common Internet applications, including streaming video, Steam and World of Warcraft. Our results show that nDPI and libprotoident provide the highest accuracy among the evaluated traffic classifiers, whereas L7 Filter is unreliable and should not be used as a source of ground truth.