In-band Network Telemetry: A Survey

This paper focuses on facilitating state-of-the-art applications of big data analytics &#x0028 BDA &#x0029 architectures and infrastructures to telecommunications &#x0028 telecom &#x0029 industrial sector. Telecom companies are dealing with terabytes to petabytes of data on a daily basis. IoT applications in telecom are further contributing to this data deluge. Recent advances in BDA have exposed new opportunities to get actionable insights from telecom big data. These benefits and the fast-changing BDA technology landscape make it important to investigate existing BDA applications to telecom sector. For this, we initially determine published research on BDA applications to telecom through a systematic literature review through which we filter 38 articles and categorize them in frameworks, use cases, literature reviews, white papers and experimental validations. We also discuss the benefits and challenges mentioned in these articles. We find that experiments are all proof of concepts &#x0028 POC &#x0029 on a severely limited BDA technology stack &#x0028 as compared to the available technology stack &#x0029 , i.e., we did not find any work focusing on full-fledged BDA implementation in an operational telecom environment. To facilitate these applications at research-level, we propose a state-of-the-art lambda architecture for BDA pipeline implementation &#x0028 called LambdaTel &#x0029 based completely on open source BDA technologies and the standard Python language, along with relevant guidelines. We discovered only one research paper which presented a relatively-limited lambda architecture using the proprietary AWS cloud infrastructure. We believe LambdaTel presents a clear roadmap for telecom industry practitioners to implement and enhance BDA applications in their enterprises.

Big data analytics in telecommunications: literature review and architecture recommendations

Encrypted traffic classification based on Gaussian mixture models and Hidden Markov Models

The rapid usage of the Internet for the last few decades has lead to the deployment of high-speed networks in commercial and educational institutions. As network traffic is increasing, security challenges are also increasing in the high-speed network. Although the Intrusion Detection System (IDS) has a significant role in spotting potential attacks, the heavy traffic flow causes severe technical challenges relating to monitoring and detecting the network activities. Moreover, the devastating nature of the Distributed Denial-of-Service (DDoS) attack draws out as a significant cyber-attack regardless of the emergence of Software Defined Network (SDN) architecture. This paper proposes a novel framework to address the performance issues of IDS and the design issues of SDN about DDoS attacks by incorporating intelligence in the data layer using Data Plane Development Kit (DPDK) in the SDN architecture. This novel framework is named as DPDK based DDoS Detection (D3) framework, since DPDK provides fast packet processing and monitoring in the data plane. Moreover, the statistical anomaly detection algorithm implemented in the data plane as Virtual Network Function (VNF) using DPDK offers fast detection of DDoS attacks. The experimental results of the D3 framework guarantee both efficiency and effect of the novel IDS framework. The publicly available CIC DoS datasets also ensure the detection effect of a single statistical anomaly detection algorithm against the DDoS attack.

/pdf/an-efficient-ids-framework-for-ddos-attacks-in-sdn-1skai3obzc.pdf

An Efficient IDS Framework for DDoS Attacks in SDN Environment

Bandwidth reservation can effectively improve the service quality for data transfers because of dedicated network resources. However, it is difficult to achieve a desired tradeoff between resource utilization and reliable bandwidth guarantees for data transfers with time-varying traffic. In this article, we study a novel bandwidth reservation solution based on distributed traffic monitoring and control for applications that require reliable bandwidth guarantees. In the proposed solution, designated bandwidth is allocated for an application in advance according to its maximum traffic peak, and idle reserved bandwidth resources are dynamically shared according to regular traffic. Dynamic resource sharing evidently improves resource utilization and effectively eliminates the potential congestion caused by sudden traffic bursts. To ensure that the congestion that occurs occasionally can dissipate rapidly, our solution monitors and manages traffic by a distributed monitoring and control strategy. Hence, we study a delay-constrained and proxy-assisted traffic monitoring structure construction problem and propose an algorithm to solve it. The proposed algorithm can also be used to build a delay-constrained traffic control structure. In addition to the above algorithm, we propose a dynamic traffic control algorithm that can achieve a desirable tradeoff between resource utilization and congestion avoidance capability. 

Elastic and Reliable Bandwidth Reservation Based on Distributed Traffic Monitoring and Control

An approach for SDN traffic monitoring based on big data techniques

A traffic matrix (TM) is a source of critical traffic throughput information for traffic engineering activities and network management tasks such as traffic prediction, capacity planning, network provisioning, and anomaly detection. However, estimating TM poses several challenges for network engineers. One of the challenges is that traffic data statistics are constantly changing, and their aggregation for real-time monitoring becomes a difficult task. This paper presents a near real-time TM estimation approach for OpenFlow (OF) networks. It makes use of Big Data techniques based on MapReduce operations to tackle the aggregation problem. The proposed method uses traffic data statistics collected from OF switches through an SDN controller as input and aggregates these data in a Big Data streaming processing environment. This paper explores the benefits of the distributed MapReduce computing model to provide an estimate of the TM for all origin-destination (OD) pairs of hosts in the network in two ways: 1) the accumulated throughput and 2) the throughput between two sequential TM estimates. This procedure enables network engineers to monitor the behavior and evolution of the throughput on each OD pair in the network and on each link in the path between each OD pair. The generated TM is persisted in a NoSQL database and can be made available for a variety of network traffic monitoring applications. The results of the simulations show the potential of the proposed MapReduce approach for TM estimation.

/pdf/a-mapreduce-approach-for-traffic-matrix-estimation-in-sdn-4yu2mg72k0.pdf

A MapReduce Approach for Traffic Matrix Estimation in SDN

As blockchain technology matures, more sophisticated solutions arise regarding complex problems. Blockchain continues to spread towards various niches such as government, IoT, energy, and environmental industries. One often overlooked opportunity for blockchain is the social responsibility sector. Presented in this paper is a permissioned blockchain model that enables enterprises to come together and cooperate to optimize their environmental and societal impacts. This is made possible through a private or permissioned blockchain. Permissioned blockchains are blockchain networks where all the participants are known and trust relationships among them can be fostered more smoothly. An example of what a permissioned blockchain would look like is described in this paper as well as its implementation, achieved using Hyperledger Fabric, which is a business-oriented blockchain framework. This study touches on the benefits available for companies that are willing to engage in socially responsible causes through blockchain. It states in what ways a permissioned blockchain can bring together businesses on common ground to increase their reach and provide better customer service. Finally, a use case is provided to bring to life a real-world situation where blockchain use improves service quality for all the parties involved, both the companies and their customers.

/pdf/a-blockchain-approach-to-social-responsibility-2zm8ry6c9w.pdf

Wander Queiroz

Papers

An approach for SDN traffic monitoring based on big data techniques

A MapReduce Approach for Traffic Matrix Estimation in SDN

A Blockchain Approach to Social Responsibility