Datacenters provide cost-effective and flexible access to scalable compute and storage resources necessary for today’s cloud computing needs. A typical datacenter is made up of thousands of servers connected with a large network and usually managed by one operator. To provide quality access to the variety of applications and services hosted on datacenters and maximize performance, it deems necessary to use datacenter networks effectively and efficiently. Datacenter traffic is often a mix of several classes with different priorities and requirements. This includes user-generated interactive traffic, traffic with deadlines, and long-running traffic. To this end, custom transport protocols and traffic management techniques have been developed to improve datacenter network performance. In this tutorial paper, we review the general architecture of datacenter networks, various topologies proposed for them, their traffic properties, general traffic control challenges in datacenters and general traffic control objectives. The purpose of this paper is to bring out the important characteristics of traffic control in datacenters and not to survey all existing solutions (as it is virtually impossible due to massive body of existing research). We hope to provide readers with a wide range of options and factors while considering a variety of traffic control mechanisms. We discuss various characteristics of datacenter traffic control, including management schemes, transmission control, traffic shaping, prioritization, load balancing, multipathing, and traffic scheduling. Next, we point to several open challenges as well as new and interesting networking paradigms. At the end of this paper, we briefly review inter-datacenter networks that connect geographically dispersed datacenters, which have been receiving increasing attention recently and pose interesting and novel research problems. To measure the performance of datacenter networks, different performance metrics have been used, such as flow completion times, deadline miss rate, throughput, and fairness. Depending on the application and user requirements, some metrics may need more attention. While investigating different traffic control techniques, we point out the tradeoffs involved in terms of costs, complexity, and performance. We find that a combination of different traffic control techniques may be necessary at particular entities and layers in the network to improve the variety of performance metrics. We also find that despite significant research efforts, there are still open problems that demand further attention from the research community.

Datacenter Traffic Control: Understanding Techniques and Tradeoffs

This paper studies the effectiveness of TCP pacing in a data center setting. TCP senders inject bursts of packets into the network at the beginning of each round-trip time. These bursts stress the network queues which may cause loss, reduction in throughput and increased latency. Such undesirable effects become more pronounced in data center environments where traffic is bursty in nature and buffer sizes are small. TCP pacing is believed to reduce the burstiness of TCP traffic and to mitigate the impact of small buffering in routers. Unfortunately, current research literature has not always agreed on the overall benefits of pacing. In this paper, we present a model for the effectiveness of pacing. Our model demonstrates that for a given buffer size, as the number of concurrent flows are increased beyond a Point of Inflection (PoI), non-paced TCP outperforms paced TCP. We present a lower bound for the PoI and argue that increasing the number of concurrent flows beyond the PoI, increases inter-flow burstiness of paced packets and diminishes the effectiveness of pacing.

/pdf/tcp-pacing-in-data-center-networks-4m2tgdsvj6.pdf

TCP Pacing in Data Center Networks

Reducing the power consumption of network interfaces contributes to lowering the overall power needs of the compute and communication infrastructure. Most modern Ethernet interfaces can operate at one of several data rates. In this paper, we present Queue Length Based Rate Adaptation (QLBRA), which can dynamically adapt the link rate for Ethernet interfaces at runtime using existing Ethernet standards. An implementation of the proposed rate adaptation functionality is demonstrated at runtime on a NetFPGA platform. Our results show that the rate adaptation approach can achieve significant energy savings and at the same time improve the throughput of TCP traffic due to the effect of packet pacing.

/pdf/saving-energy-and-improving-tcp-throughput-with-rate-3uz4t7jk40.pdf

Saving energy and improving TCP throughput with rate adaptation in Ethernet

Packet losses in the network have a considerable performance impact on transport-layer throughput. For reliable data transfer, lost packets require retransmissions and thus cause very long delays. This tail of the packet delay distribution causes performance problems. There are several approaches to trading off networking resources up-front to reduce long delays for some packets (e.g., forward error correction, network coding). We propose packet pacing as an alternative that changes traffic characteristics favorably by adding intentional delay in packet transmissions. This intentional delay counters the principle of best effort but can reduce the burstiness of traffic and improve overall network operation - in particular in network with small packet buffers. As a result, pacing improves transport-layer performance, providing a tradeoff example where small amounts of additional delay can significantly increase connection bandwidth. We present a Queue Length Based Pacing (QLBP) algorithm that paces network traffic using a single queue and that can be implemented with small computational and memory overhead. We present a detailed analysis on delay bounds and the quantitative impact of QLBP pacing on network traffic. Through simulation, we show how the proposed pacing technique can improve connection throughput in small-buffer networks.

/pdf/delaying-transmissions-in-data-communication-networks-to-3haygs4sqb.pdf

Delaying Transmissions in Data Communication Networks to Improve Transport-Layer Performance

For the optical packet-switching routers to be widely deployed in the Internet, the size of packet buffers on routers has to be significantly small. Such small-buffer networks rely on traffic with low levels of burstiness to avoid buffer overflows and packet losses. We present a pacing system that proactively shapes traffic in the edge network to reduce burstiness. Our queue length based pacing uses an adaptive pacing on a single queue and paces traffic indiscriminately where deployed. In this work, we show through analysis and simulation that this pacing approach introduces a bounded delay and that it effectively reduces traffic burstiness. We also show that it can achieve higher throughput than end-system based pacing.

/pdf/a-practical-on-line-pacing-scheme-at-edges-of-small-buffer-4seun1wqub.pdf

A Practical On-line Pacing Scheme at Edges of Small Buffer Networks

The demand for more bandwidth has lead to proposals for an all-optical network core. Due to inherent constraints of optical technology, only routers with small packet buffers are feasible to be implemented. In order to ensure efficient operation of such small-buffer networks, it is necessary to ensure that traffic is less bursty than in conventional networks. We propose a novel packet pacing mechanism that can smooth traffic bursts. Our theoretical analysis shows that our pacing scheme can guarantee that queue length of routers is BIBO stable. Experimental results from our prototype implementation show the effectiveness of our pacer on in terms of reduced network congestion and improving network throughput.

/pdf/practical-packet-pacing-in-small-buffer-networks-semmdsgjo8.pdf

Practical Packet Pacing in Small-Buffer Networks

The exchange of topology information is a potential attack target in mobile ad-hoc networks. To provide an intrinsic security mechanism, it is possible to validate topology advertisements in the control plane against records of the path taken by transmission in the data plane. In this context, we provide a discussion of different path recording mechanisms. We evaluate their performance in terms of packet overhead and reconstruction complexity.

/pdf/topology-reconstruction-via-path-recording-in-secure-manet-1u8b8ubjmn.pdf

Y. S. Hanay

Papers

Practical Packet Pacing in Small-Buffer Networks

Topology reconstruction via path recording in secure MANET