In this paper, we show that up to hundreds of software load balancer (SLB) servers can be replaced by a single modern switching ASIC, potentially reducing the cost of load balancing by over two orders of magnitude. Today, large data centers typically employ hundreds or thousands of servers to load-balance incoming traffic over application servers. These software load balancers (SLBs) map packets destined to a service (with a virtual IP address, or VIP), to a pool of servers tasked with providing the service (with multiple direct IP addresses, or DIPs). An SLB is stateful, it must always map a connection to the same server, even if the pool of servers changes and/or if the load is spread differently across the pool. This property is called per-connection consistency or PCC. The challenge is that the load balancer must keep track of millions of connections simultaneously.Until recently, it was not possible to implement a load balancer with PCC in a merchant switching ASIC, because high-performance switching ASICs typically can not maintain per-connection states with PCC. Newer switching ASICs provide resources and primitives to enable PCC at a large scale. In this paper, we explore how to use switching ASICs to build much faster load balancers than have been built before. Our system, called SilkRoad, is defined in a 400 line P4 program and when compiled to a state-of-the-art switching ASIC, we show it can load-balance ten million connections simultaneously at line rate.

https://dl.acm.org/doi/pdf/10.1145/3098822.3098824

SilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs

Load-balancing algorithms in cloud computing

Review: A survey on content-centric technologies for the current Internet: CDN and P2P solutions

Topic indexing is the task of identifying the main topics covered by a document. These are useful for many purposes: as subject headings in libraries, as keywords in academic publications and as tags on the web. Knowing a document’s topics helps people judge its relevance quickly. However, assigning topics manually is labor intensive. This thesis shows how to generate them automatically in a way that competes with human performance. Three kinds of indexing are investigated: term assignment, a task commonly performed by librarians, who select topics from a controlled vocabulary; tagging, a popular activity of web users, who choose topics freely; and a new method of keyphrase extraction, where topics are equated to Wikipedia article names. A general two-stage algorithm is introduced that first selects candidate topics and then ranks them by significance based on their properties. These properties draw on statistical, semantic, domain-specific and encyclopedic knowledge. They are combined using a machine learning algorithm that models human indexing behavior from examples. This approach is evaluated by comparing automatically generated topics to those assigned by professional indexers, and by amateurs. We claim that the algorithm is “human-competitive” because it chooses topics that are as consistent with those assigned by humans as their topics are with each other. The approach is generalizable, requires little training data and applies across different domains and languages.

Human-competitive automatic topic indexing

In a call center, arriving customers must be routed to available servers, and servers that have just become available must be scheduled to help waiting customers. These dynamic routing and scheduling decisions are very difficult, because customers have different needs and servers have different skill levels. A further complication is that it is preferable that these decisions are made blindly; that is, they depend only on the system state and not on system parameter information such as call arrival rates and service speeds. This is because this information is generally not known with certainty. Ideally, a dynamic control policy for making routing and scheduling decisions balances customer and server needs by keeping customer delays low but still fairly dividing the workload amongst the various servers. In this paper, we propose a blind dynamic control policy for parallel-server systems with multiple customer classes and server pools that is based on the number of customers waiting and the number of agents idling.

We show that in the Halfin-Whitt many-server heavy-traffic limiting regime, our proposed blind policy performs extremely well when the objective is to minimize customer holding costs subject to “server fairness,” as defined by how the system idleness is divided among servers. To do this, we formulate an approximating diffusion control problem DCP and compare the performance of the nonblind DCP solution to a feasible policy for the DCP that is blind. We establish that the increase in the DCP objective function value is small over a wide range of parameter values. We then use simulation to validate that a small increase in the DCP objective function value is indicative of our proposed blind policy performing very well.

Blind Fair Routing in Large-Scale Service Systems with Heterogeneous Customers and Servers

Load balancing is a standard feature of EFI XF 3.1 and later and is described in the manual provided with the product. However, in EFI XF 3.0, load balancing is not an officially supported feature and can only be achieved by manual ticket editing. Manual ticket editing is only recommended for experienced users. This document describes the steps necessary to achieve load balancing in EFI XF 3.0. NOTE: This method of load balancing cannot be applied to versions of EFI XF prior to 3.0.

R. McConnell

Papers

Load Balancing