To be agile and cost effective, data centers should allow dynamic resource allocation across large server pools. In particular, the data center network should enable any server to be assigned to any service. To meet these goals, we present VL2, a practical network architecture that scales to support huge data centers with uniform high capacity between servers, performance isolation between services, and Ethernet layer-2 semantics. VL2 uses (1) flat addressing to allow service instances to be placed anywhere in the network, (2) Valiant Load Balancing to spread traffic uniformly across network paths, and (3) end-system based address resolution to scale to large server pools, without introducing complexity to the network control plane. VL2's design is driven by detailed measurements of traffic and fault data from a large operational cloud service provider. VL2's implementation leverages proven network technologies, already available at low cost in high-speed hardware implementations, to build a scalable and reliable network architecture. As a result, VL2 networks can be deployed today, and we have built a working prototype. We evaluate the merits of the VL2 design using measurement, analysis, and experiments. Our VL2 prototype shuffles 2.7 TB of data among 75 servers in 395 seconds - sustaining a rate that is 94% of the maximum possible.

/pdf/vl2-a-scalable-and-flexible-data-center-network-dkbtxdift1.pdf

VL2: a scalable and flexible data center network

To be agile and cost effective, data centers must allow dynamic resource allocation across large server pools. In particular, the data center network should provide a simple flat abstraction: it should be able to take any set of servers anywhere in the data center and give them the illusion that they are plugged into a physically separate, noninterfering Ethernet switch with as many ports as the service needs. To meet this goal, we present VL2, a practical network architecture that scales to support huge data centers with uniform high capacity between servers, performance isolation between services, and Ethernet layer-2 semantics. VL2 uses (1) flat addressing to allow service instances to be placed anywhere in the network, (2) Valiant Load Balancing to spread traffic uniformly across network paths, and (3) end system--based address resolution to scale to large server pools without introducing complexity to the network control plane. VL2's design is driven by detailed measurements of traffic and fault data from a large operational cloud service provider. VL2's implementation leverages proven network technologies, already available at low cost in high-speed hardware implementations, to build a scalable and reliable network architecture. As a result, VL2 networks can be deployed today, and we have built a working prototype. We evaluate the merits of the VL2 design using measurement, analysis, and experiments. Our VL2 prototype shuffles 2.7 TB of data among 75 servers in 395 s---sustaining a rate that is 94% of the maximum possible.

Routers built around a single-stage crossbar and a centralized scheduler do not scale, and (in practice) do not provide the throughput guarantees that network operators need to make efficient use of their expensive long-haul links. In this paper we consider how optics can be used to scale capacity and reduce power in a router. We start with the promising load-balanced switch architecture proposed by C-S. Chang. This approach eliminates the scheduler, is scalable, and guarantees 100% throughput for a broad class of traffic. But several problems need to be solved to make this architecture practical: (1) Packets can be mis-sequenced, (2) Pathological periodic traffic patterns can make throughput arbitrarily small, (3) The architecture requires a rapidly configuring switch fabric, and (4) It does not work when linecards are missing or have failed. In this paper we solve each problem in turn, and describe new architectures that include our solutions. We motivate our work by designing a 100Tb/s packet-switched router arranged as 640 linecards, each operating at 160Gb/s. We describe two different implementations based on technology available within the next three years.

/pdf/scaling-internet-routers-using-optics-1tvpfgckv2.pdf

Scaling internet routers using optics

On-chip network is becoming critical to the scalability of future many-core architectures. Recently, nanophotonics has been proposed for on-chip networks because of its low latency and high bandwidth. However, nanophotonics has relatively high static power consumption, which can lead to inefficient architectures. In this work, we propose FlexiShare — a nanopho-tonic crossbar architecture that minimizes static power consumption by fully sharing a reduced number of channels across the network. To enable efficient global sharing, we decouple the allocation of the channels and the buffers, and introduce novel photonic token-stream mechanism for channel arbitration and credit distribution The flexibility of FlexiShare introduces additional router complexity and electrical power consumption. However, with the reduced number of optical channels, the overall power consumption is reduced without loss in performance. Our evaluation shows that the proposed token-stream arbitration applied to a conventional crossbar design improves network throughput by 5.5× under permutation traffic. In addition, FlexiShare achieves similar performance as a token-stream arbitrated conventional crossbar using only half the amount of channels under balanced, distributed traffic. With the extracted trace traffic from MineBench and SPLASH-2, FlexiShare can further reduce the amount of channels by up to 87.5%, while still providing better performance — resulting in up to 72% reduction in power consumption compared to the best alternative.

/pdf/flexishare-channel-sharing-for-an-energy-efficient-3qtc5feyth.pdf

FlexiShare: Channel sharing for an energy-efficient nanophotonic crossbar

The ever-increasing bandwidth requirements of modern datacenters have led researchers to propose networks based upon optical circuit switches, but these proposals face significant deployment challenges. In particular, previous proposals dynamically configure circuit switches in response to changes in workload, requiring network-wide demand estimation, centralized circuit assignment, and tight time synchronization between various network elements--- resulting in a complex and unwieldy control plane. Moreover, limitations in the technologies underlying the individual circuit switches restrict both the rate at which they can be reconfigured and the scale of the network that can be constructed.We propose RotorNet, a circuit-based network design that addresses these two challenges. While RotorNet dynamically reconfigures its constituent circuit switches, it decouples switch configuration from traffic patterns, obviating the need for demand collection and admitting a fully decentralized control plane. At the physical layer, RotorNet relaxes the requirements on the underlying circuit switches---in particular by not requiring individual switches to implement a full crossbar---enabling them to scale to 1000s of ports. We show that RotorNet outperforms comparably priced Fat Tree topologies under a variety of workload conditions, including traces taken from two commercial datacenters. We also demonstrate a small-scale RotorNet operating in practice on an eight-node testbed.

https://dl.acm.org/doi/pdf/10.1145/3098822.3098838

RotorNet: A Scalable, Low-complexity, Optical Datacenter Network

Load balanced Birkhoff-von Neumann switches, part II: multi-stage buffering

In [2], we proposed the load balanced Birkhoff-von Neumann switch with one-stage buffering (see Figure 1). Such a switch consists of two stages of crossbar switching fabrics and one stage of buffering. The buffer at the input port of the second stage uses the Virtual Output Queueing (VOQ) technique to solve the problem of head-of-line blocking. In such a switch, packets are of the same size. Also, time is slotted and synchronized so that exactly one packet can be transmitted within a time slot. In a time slot, both crossbar switches set up connection patterns corresponding to permutation matrices that are periodically generated from a one-cycle permutation matrix.The reasoning behind such a switch architecture is as follows: since the connection patterns are periodic, packets from the same input port of the first stage are distributed in a round-robin fashion to the second stage according to their arrival times. Thus, the first stage performs load balancing for the incoming traffic. As the traffic coming into the second stage is load balanced, it suffices to use simple periodic connection patterns to perform switching at the second stage. This is shown in [2] as a special case of the original Birkhoff-von Neumann decomposition used in [1]. There are several advantages of using such an architecture, including scalability, low hardware complexity, 100% throughput, low average delay in heavy load and bursty traffic, and efficient buffer usage. However, the main drawback of the load balanced Birkhoff-von Neumann switch with one-stage buffering is that packets might be out of sequence.The main objective of this paper is to solve the out-of-sequence problem that occurs in the load balanced Birkhoff-von Neumann switch with one-stage buffering. One quick fix is to add a resequencing-and-output buffer after the second stage. However, as packets are distributed according to their arrival times at the first stage, there is no guarantee on the size of the resequencing-and-output buffer to prevent packet losses. For this, one needs to distributed packets according to their flows, as indicated in the paper by Iyer and McKeown [5]. This is done by adding a flow splitter and a load-balancing buffer in front of the first stage (see Figure 2). For an N x N switch, the load-balancing buffer at each input port of the first stage consists of N virtual output queues (VOQ) destined for the N output ports of that stage. Packets form the same flow are split in the round-robin fashion to the N virtual output queues and scheduled under the First Come First Served (FCFS) policy. By so doing, load balancing can be achieved for each flow as packets from the same flow are split almost evenly to the input ports of the second stage. More importantly, as pointed out in [5], the delay and the buffer size of the load-balancing buffer are bounded by constants that only depend on the size of the switch and the number of flows. The resequencing-and-output buffer after the second stage not only performs resequencing to keep packets in sequence, but also stores packets waiting for transmission from the output links.In this paper, we consider a traffic model with multicasting flows. This is a more general model than the point-to-point traffic model in [5]. A multicasting flow is stream of packets that has one common input and a set of common outputs. For the multicasting flows, fanout splitting (see e.g., [4]) is performed at the central buffers (the VOQ in front of the second stage). The central buffers are assumed to be infinite so that no packets are lost in the switch. We consider two types of scheduling policies in the central buffers: the FCFS policy and the Earliest Deadline First (EDF) policy. For the FCFS policy, a jitter control mechanism, is added in the VOQ in front of the second stage. Such a jitter control mechanism delays every packet to its maximum delay at the first stage so that the flows entering the second stage are simply time-shifted flows of the original ones. Our main result for the FCFS scheme with jitter controls is the following theorem. The proof of Theorem 1 is shown in the full report [3].Theorem 1Suppose that all the buffers are empty at time 0. Then the followings hold for FCFS scheme with jitter control.(i) The end-to-end delay for a packet through our switch with multi-stage buffering is bounded above by the sum of the delay through the corresponding FCFS output-buffered switch and N Lmax + (N + 1)Mmax, where Lmax(resp. Mmax) is the maximum number of flows at an input (resp. output) port.(ii) The load-balancing buffer at an input port of the first stage is bounded above by N Lmax.(iii) The delay through the load-balancing buffer at an input port of the first stage is bounded above by N Lmax.(iv) The resequencing-and-output buffer at an output port of the second stage is bounded above (N + 1)Mmax.In the EDF scheme (see Figure 3), every packet is assigned a deadline that is the departure time from the corresponding FCFS output-buffered switch. Packets are scheduled according to their deadlines in the central buffers. For the EDF scheme, there is no need to implement the jitter control mechanism in the FCFS scheme. As such, average packet delay can be greatly reduced. However, as there is no jitter control, one might need a larger resequencing buffer than that in the FCFS scheme with jitter control. Since the first stage is the same as that in the FCFS scheme, both the delay and the buffer size of the load-balancing buffer are still bounded by N Lmax. Moreover, we show the following theorem for the EDF scheme. Its proof is given in the full report [3].Theorem 2Suppose that all the buffers are empty at time 0. Then the followings hold for the EDF scheme.(i) The end-to-end delay for a packet through our switch with multi-stage buffering is bounded above by the sum of the delay through the corresponding FCFS output-buffered switch and N (Lmax + Mmax).(ii) The resequencing-and-output buffer at an output port of the second stage is bounded above N (Lmax+Mmax).Computing the departure times from the corresponding FCFS output-buffered switch needs global information of all the inputs. A simple way is to use the packet arrival times as deadlines. Then the EDF scheme based on arrival times yields the same departure order except those packets that arrives at same time. Since there are at most Mmax packets that can arrive at the same time to an output port of the corresponding output-buffered switch, the end-to-end delay for a packet through the multi-stage switch using arrival times as deadlines is bounded above by the sum of the delay through the corresponding FCFS output-buffered switch and N Lmax+(N+1)Mmax. Also, the resequencing-and-output buffer at an output port of the second stage in this case is bounded above N Lmax + (N + 1)Mmax.

Load balanced Birkhoff-von Neumann switches with resequencing

In this paper, we consider a multicasting model that uses incremental forward error correction (FEC). In this model, there is one sender and r/sup n/ receivers. The sender uses an ideal (n,n(1-p),np) FEC code to code a group of n(1-p) data packets with additional np redundant packets so that any set of n(1-p) packets received by a receiver can be used to recover the original n(1-p) data packets. Packets to the receivers are lost independently with probability q. For this model, we prove several strong laws of large numbers for the asymptotic throughput as n /spl rarr/ /spl infin/. The asymptotic throughput is characterized by the unique solution of an equation in terms of p, q, and r. These strong laws not only provide theoretical justification for several important observations made in the literature, but also provide insights that might have impact on future design of multicasting protocols.

Ching-Ming Lien

Papers

Load balanced Birkhoff-von Neumann switches, part II: multi-stage buffering

Load balanced Birkhoff-von Neumann switches with resequencing

On the throughput of multicasting with incremental forward error correction