Recent proposals have employed optical circuit switching (OCS) to reduce the cost of data center networks. However, the relatively slow switching times (10--100 ms) assumed by these approaches, and the accompanying latencies of their control planes, has limited its use to only the largest data center networks with highly aggregated and constrained workloads. As faster switch technologies become available, designing a control plane capable of supporting them becomes a key challenge.In this paper, we design and implement an OCS prototype capable of switching in 11.5 us, and we use this prototype to expose a set of challenges that arise when supporting switching at microsecond time scales. In response, we propose a microsecond-latency control plane based on a circuit scheduling approach we call Traffic Matrix Scheduling (TMS) that proactively communicates circuit assignments to communicating entities so that circuit bandwidth can be used efficiently.

/pdf/integrating-microsecond-circuit-switching-into-the-data-58q77bdiqn.pdf

Integrating microsecond circuit switching into the data center

Large companies like Facebook, Google, and Microsoft as well as a number of small and medium enterprises daily process massive amounts of data in batch jobs and in real time applications. This generates high network traffic, which is hard to support using traditional, oversubscribed, network infrastructures. To address this issue, several novel network topologies have been proposed, aiming at increasing the bandwidth available in enterprise clusters.

We observe that in many of the commonly used workloads, data is aggregated during the process and the output size is a fraction of the input size. This motivated us to explore a different point in the design space. Instead of increasing the bandwidth, we focus on decreasing the traffic by pushing aggregation from the edge into the network.

We built Camdoop, a MapReduce-like system running on CamCube, a cluster design that uses a direct-connect network topology with servers directly linked to other servers. Camdoop exploits the property that CamCube servers forward traffic to perform in-network aggregation of data during the shuffle phase. Camdoop supports the same functions used in MapReduce and is compatible with existing MapReduce applications. We demonstrate that, in common cases, Camdoop significantly reduces the network traffic and provides high performance increase over a version of Camdoop running over a switch and against two production systems, Hadoop and Dryad/DryadLINQ.

/pdf/camdoop-exploiting-in-network-aggregation-for-big-data-ql4sjwfhlu.pdf

Camdoop: exploiting in-network aggregation for big data applications

A method and system for enabling Non-Volatile Memory express (NVMe) for accessing remote solid state drives (SSDs) (or other types of remote non-volatile memory) over the Ethernet or other networks. An extended NVMe controller is provided for enabling CPU to access remote non-volatile memory using NVMe protocol. The extended NVMe controller is implemented on one server for communication with other servers or non-volatile memory via Ethernet switch. The NVMe protocol is used over the Ethernet or similar networks by modifying it to provide a special NVM-over-Ethernet frame.

NVM express controller for remote access of memory and I/O over ethernet-type networks

A computer system according to the present invention includes a managing unit which outputs a plurality of virtual networks managed by a plurality of controllers in a visually perceivable form with the plurality of virtual networks combined, on the basis of topology data of the virtual networks, the topology data being generated based on communication routes. This enables centralized management of the whole of a virtual network controlled by a plurality of controllers which use an OpenFlow technology.

Computer system and virtual network visualization method

In one embodiment, method of wrapping photonic packets includes receiving, by a node, a first packet and receiving, by the node, a second packet. The method also includes concatenating the first packet and the second packet to produce a concatenated frame, where concatenating the first packet and the second packet includes removing an inter-packet-gap (IPG) between the first packet and the second packet and converting the concatenated frame to a photonic frame, where the concatenated frame is an electrical frame.

System and Method for Photonic Switching

Data center network architectures that can reduce the cost and complexity of data center networks. The data center network architectures can employ optical network topologies and optical nodes to efficiently allocate bandwidth within the data center networks, while reducing the physical interconnectivity requirements of the data center networks. The data center network architectures also allow computing resources within data center networks to be controlled and provisioned based at least in part on a combined network topology and application component topology, thereby enhancing overall application program performance.

Data center network architecture

Systems and methods of affinity modeling in data center networks that allow bandwidth to be efficiently allocated with the data center networks, while reducing the physical interconnectivity requirements of the data center networks. Such systems and methods of affinity modeling in data center networks further allow computing resources within the data center networks to be controlled and provisioned based at least in part on the network topology and an application component topology, thereby enhancing overall application program performance.

Affinity modeling in a data center network

Data center networks that employ optical network topologies and optical nodes to efficiently allocate bandwidth within the data center networks, while reducing the physical interconnectivity requirements of the data center networks. Such data center networks provide a hierarchy of control for controlling and provisioning computing resources within the data center networks based at least in part on the network topology and an application component topology, thereby enhancing overall application program performance.

Hierarchy of control in a data center network

Data center networks that employ optical network topologies and optical nodes to efficiently allocate bandwidth within the data center networks, while reducing the physical interconnectivity requirements of the data center networks. Such data center networks employ at least one central controller for controlling and provisioning computing resources within the data center networks based at least in part on the network topology and an application component topology, thereby enhancing overall application program performance.

Control and provisioning in a data center network with at least one central controller

Systems and methods of affinity modeling in data center networks that allow bandwidth to be efficiently allocated within the data center networks, while reducing the physical interconnectivity requirements of the data center networks. Such systems and methods of affinity modeling in data center networks further allow computing resources within the data center networks to be controlled and provisioned based at least in part on the network topology and an application component topology, thereby enhancing overall application program performance. Using an affinity topology describing requirements for communications between applications and a network topology, network nodes are configured to satisfy multicast dispersion and latency requirements associated with communications between applications.

Anand Srinivas

Papers

Data center network architecture

Affinity modeling in a data center network

Hierarchy of control in a data center network

Control and provisioning in a data center network with at least one central controller

Configuring a computer network to satisfy multicast dispersion and latency requirements using affinity and network topologies