Cloud services have recently started undergoing a major shift from monolithic applications, to graphs of hundreds or thousands of loosely-coupled microservices. Microservices fundamentally change a lot of assumptions current cloud systems are designed with, and present both opportunities and challenges when optimizing for quality of service (QoS) and cloud utilization. In this paper we explore the implications microservices have across the cloud system stack. We first present DeathStarBench, a novel, open-source benchmark suite built with microservices that is representative of large end-to-end services, modular and extensible. DeathStarBench includes a social network, a media service, an e-commerce site, a banking system, and IoT applications for coordination control of UAV swarms. We then use DeathStarBench to study the architectural characteristics of microservices, their implications in networking and operating systems, their challenges with respect to cluster management, and their trade-offs in terms of application design and programming frameworks. Finally, we explore the tail at scale effects of microservices in real deployments with hundreds of users, and highlight the increased pressure they put on performance predictability.

https://dl.acm.org/doi/pdf/10.1145/3297858.3304013

An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud & Edge Systems

A switch, switched architecture and process for transferring data through an FCAL switch is disclosed. The switch uses multiple switch control circuits each coupled to one FCAL network and all connected to a crossbar switch. The switch control circuits are coupled together by a protocol bus for coordination purposes. Local conversations can occur on each FCAL loop and crossing conversations through the switch can occur concurrently. The OPN primitive is used to establish the connection before any data is transferred thereby eliminating the need for buffer memory in the switch control circuits. The destination address of each OPN is used to address a lookup table in each switch control circuit to determine if the destination node is local. If not, the destination is looked up and a connection request made on the protocol bus. If the remote port is not busy, it sends a reply which causes both ports to establish a data path through the backplane crossbar switch.

Fibre channel arbitrated loop bufferless switch circuitry to increase bandwidth without significant increase in cost

Management of data communication between a peripheral device and host computer system is provided. A peripheral device exposes to a host computer system multiple interfaces for data communication between the peripheral device and the host computer system. The multiple interfaces are exposed over a single physical interface between the peripheral device and the host computer system, for communicating data between the peripheral device and multiple applications executing on the host computer system. The multiple interfaces can include a data collection interface facilitating collection of data from the peripheral device by an application of the multiple applications executing on the host computer system.

Managing data communication between a peripheral device and a host

A method for securing remote access to private networks includes a receiver intercepting from a data link layer a packet in a first plurality of packets destined for a first system on a private network. A filter intercepts from the data link layer a packet in a second plurality of packets transmitted from a second system on the private network, destined for an system on a second network. A transmitter in communication with the receiver and the filter performing a network address translation on at least one intercepted packet and transmitting the at least one intercepted packet to a destination.

A method and systems for securing remote access to private networks

One embodiment of the present invention provides a switch. The switch includes a port that couples to a server hosting a number of virtual machines. The switch also includes a set of virtual port grouping information and a virtual port grouping mechanism. During operation, the virtual port grouping mechanism determines whether a frame is traveling from a virtual port group to the same virtual port group. If the frame is not traveling to the same virtual port group, the virtual port grouping mechanism prevents the frame from being forwarded.

Virtual port grouping for virtual cluster switching

Modern cloud architectures rely on each server running its own networking stack to implement policies such as tunneling for virtual networks, security, and load balancing. However, these networking stacks are becoming increasingly complex as features are added and as network speeds increase. Running these stacks on CPU cores takes away processing power from VMs, increasing the cost of running cloud services, and adding latency and variability to network performance.

We present Azure Accelerated Networking (AccelNet), our solution for offloading host networking to hardware, using custom Azure SmartNICs based on FPGAs. We define the goals of AccelNet, including programmability comparable to software, and performance and efficiency comparable to hardware. We show that FPGAs are the best current platform for offloading our networking stack as ASICs do not provide sufficient programmability, and embedded CPU cores do not provide scalable performance, especially on single network flows.

Azure SmartNICs implementing AccelNet have been deployed on all new Azure servers since late 2015 in a fleet of >1M hosts. The AccelNet service has been available for Azure customers since 2016, providing consistent <15µs VM-VM TCP latencies and 32Gbps throughput, which we believe represents the fastest network available to customers in the public cloud. We present the design of AccelNet, including our hardware/software codesign model, performance results on key workloads, and experiences and lessons learned from developing and deploying AccelNet on FPGA-based Azure SmartNICs.

/pdf/azure-accelerated-networking-smartnics-in-the-public-cloud-5ce6k3swsw.pdf

Azure accelerated networking: SmartNICs in the public cloud

Offloading specific processing tasks that would otherwise be performed in a computer system's processor and memory, to a peripheral device. The computing task is then performed by the peripheral, thereby saving computer system resources for other computing tasks. In one preferred embodiment, the disclosed method is utilized in a layered network model, wherein computing tasks that are typically performed in network applications are instead offloaded to the network interface card (NIC) peripheral.

Method and computer program product for offloading processing tasks from software to hardware

An extensible virtual switch allows virtual machines to communicate with one another and optionally with other physical devices via a network. The extensible virtual switch includes an extensibility protocol binding, allowing different extensions to be added to the extensible virtual switch. The extensible virtual switch also includes a miniport driver on which the extensions are loaded, tying the lifetimes of the extensions to the lifetime of the extensible virtual switch.

Virtual switch extensibility

Techniques for efficient and secure implementation of network policies in a network interface controller (NIC) in a host computing device operating a virtualized computing environment. In some embodiments, the NIC may process and forward packets directly to their destinations, bypassing a parent partition of the host computing device. In particular, in some embodiments, the NIC may store network policy information to process and forward packets directly to a virtual machine (VM). If the NIC is unable to process a packet, then the NIC may forward the packet to the parent partition. In some embodiments, the NIC may use an encapsulation protocol to transmit address information in packet headers. In some embodiments, this address information may be communicated by the MC to the parent partition via a secure channel. The NIC may also obtain, and decrypt, encrypted addresses from the VMs for routing packets, bypassing the parent partition.

Secure and efficient offloading of network policies to network interface cards

Techniques are disclosed for the non-disruptive and reliable live migration of a virtual machine (VM) from a source host to a target host, where network data is placed directly into the VM's memory. When a live migration begins, a network interface card (NIC) of the source stops placing newly received packets into the VM's memory. A virtual server driver (VSP) on the source stores the packets being processed and forces a return of the memory where the packets are stored to the NIC. When the VM has been migrated to the target, and the source VSP has transferred the stored packets to the target host, the VM resumes processing the packets, and when the VM sends messages to the target NIC that the memory associated with a processed packet is free, a VSP on the target intercepts that message, blocking the target NIC from receiving it.

Alireza Dabagh

Papers

Azure accelerated networking: SmartNICs in the public cloud

Method and computer program product for offloading processing tasks from software to hardware

Virtual switch extensibility

Secure and efficient offloading of network policies to network interface cards

Non-disruptive, reliable live migration of virtual machines with network data reception directly into virtual machines' memory