We propose a new design for highly concurrent Internet services, which we call the staged event-driven architecture (SEDA). SEDA is intended to support massive concurrency demands and simplify the construction of well-conditioned services. In SEDA, applications consist of a network of event-driven stages connected by explicit queues. This architecture allows services to be well-conditioned to load, preventing resources from being overcommitted when demand exceeds service capacity. SEDA makes use of a set of dynamic resource controllers to keep stages within their operating regime despite large fluctuations in load. We describe several control mechanisms for automatic tuning and load conditioning, including thread pool sizing, event batching, and adaptive load shedding. We present the SEDA design and an implementation of an Internet services platform based on this architecture. We evaluate the use of SEDA through two applications: a high-performance HTTP server and a packet router for the Gnutella peer-to-peer file sharing network. These results show that SEDA applications exhibit higher performance than traditional service designs, and are robust to huge variations in load.

/pdf/seda-an-architecture-for-well-conditioned-scalable-internet-25bi8q5ubv.pdf

SEDA: an architecture for well-conditioned, scalable internet services

Since many Internet applications employ a multi-tier architecture, in this paper, we focus on the problem of analytically modeling the behavior of such applications. We present a model based on a network of queues, where the queues represent different tiers of the application. Our model is sufficiently general to capture (i) the behavior of tiers with significantly different performance characteristics and (ii) application idiosyncrasies such as session-based workloads, concurrency limits, and caching at intermediate tiers. We validate our model using real multi-tier applications running on a Linux server cluster. Our experiments indicate that our model faithfully captures the performance of these applications for a number of workloads and configurations. For a variety of scenarios, including those with caching at one of the application tiers, the average response times predicted by our model were within the 95% confidence intervals of the observed average response times. Our experiments also demonstrate the utility of the model for dynamic capacity provisioning, performance prediction, bottleneck identification, and session policing. In one scenario, where the request arrival rate increased from less than 1500 to nearly 4200 requests/min, a dynamic provisioning technique employing our model was able to maintain response time targets by increasing the capacity of two of the application tiers by factors of 2 and 3.5, respectively.

/pdf/an-analytical-model-for-multi-tier-internet-services-and-its-vqq3h3j9vc.pdf

An analytical model for multi-tier internet services and its applications

Dynamic capacity provisioning is a useful technique for handling the multi-time-scale variations seen in Internet workloads. In this article, we propose a novel dynamic provisioning technique for multi-tier Internet applications that employs (1) a flexible queuing model to determine how much of the resources to allocate to each tier of the application, and (2) a combination of predictive and reactive methods that determine when to provision these resources, both at large and small time scales. We propose a novel data center architecture based on virtual machine monitors to reduce provisioning overheads. Our experiments on a forty-machine Xen/Linux-based hosting platform demonstrate the responsiveness of our technique in handling dynamic workloads. In one scenario where a flash crowd caused the workload of a three-tier application to double, our technique was able to double the application capacity within five minutes, thus maintaining response-time targets. Our technique also reduced the overhead of switching servers across applications from several minutes to less than a second, while meeting the performance targets of residual sessions.

/pdf/agile-dynamic-provisioning-of-multi-tier-internet-32pke32bfm.pdf

Agile dynamic provisioning of multi-tier Internet applications

Disclosed herein are methods, systems, and apparatuses to enable subscribers of mobile wireless communication devices to view, research, select and customize service plans; to create and manage device groups, share and set permission controls for service plans among devices in device groups; to manage communication services through graphical user interfaces; to sponsor and promote service plans; and to design, manage, and control communication services through application programming interfaces.

Service Plan Design, User Interfaces, Application Programming Interfaces, and Device Management

Device Assisted Services (DAS) for protecting network capacity is provided. In some embodiments, DAS for protecting network capacity includes monitoring a network service usage activity of the communications device in network communication; classifying the network service usage activity for differential network access control for protecting network capacity; and associating the network service usage activity with a network service usage control policy based on a classification of the network service usage activity to facilitate differential network access control for protecting network capacity.

Device-Assisted Services for Protecting Network Capacity

A method, information processing system, and computer program product manage server placement of virtual machines in an operating environment. A mapping of each virtual machine in a plurality of virtual machines to at least one server in a set of servers is determined. The mapping substantially satisfies a set of primary constraints associated with the set of servers. A plurality of virtual machine clusters is created. Each virtual machine cluster includes a set of virtual machines from the plurality of virtual machines. A server placement of one virtual machine in a cluster is interchangeable with a server placement of another virtual machine in the same cluster while satisfying the set of primary constraints. A server placement of the set of virtual machines within each virtual machine on at least one mapped server is generated for each cluster. The server placement substantially satisfies a set of secondary constraints.

Placement of virtual machines based on server cost and network cost

The increasing number of Internet users and innovativenew services such as e-commerce are placing new demands on Web servers. It is becoming essential for Web servers to provide performance isolation, have fast recovery times, and provide continuous service during overload at least to preferred customers. In this paper, we present the design and implementation of a kernel-based architecture that protects Web servers against overload by controlling the amount and rate of work entering the system. We present three mechanisms that provide admission control and service differentiation based on connection and application level information. Our basic admission control mechanism, TCP SYN policing, limits the acceptance rate of new requests based on the connection attributes. The second mechanism,prioritized listen queue, supports different service classes by reordering the listen queue of a server socket based on the priorities of the incoming connection requests. Third, we present URL-based connection control that uses application-level information such as URLs and cookies to define priorities and rate control policies. We have implemented these mechanisms in AIX 5.0. Through numerous experiments we demonstrate the effectiveness of these mechanisms in achieving the desired degree of service differentiation during overload. We also show that the kernel mechanisms are more efficient and scalable than application level controls implemented in the Web server.

Kernel Mechanisms for Service Differentiation in Overloaded Web Servers

Mechanisms are provided for implementing a multi-root PCI manager (MR-PCIM) in a multi-root I/O virtualization management partition (MR-IMP) to control the shared functionality of an multi-root I/O virtualization (IOV) enabled switch fabric and multi-root IOV enabled I/O adapter (IOA) through the base functions (BF) of the switches and IOAs. A hypervisor provides device-independent facilities to the code running in the I/O Virtualization Management Partition (IMP), Multi-Root (MR)-IMP and client partitions. The MR-IMP may include device specific code without the hypervisor needing to sacrifice its size, robustness, and upgradeability. The hypervisor provides the virtual intermediary functionally for the sharing and control of the switch and IOA's control functions.

Multi-root I/O virtualization using separate management facilities of multiple logical partitions

A hypervisor, during device discovery, has code which can examine the south-side management data structure in an adapter's configuration space and determine the type of device which is being configured. The hypervisor may copy the south-side management data structure to a hardware management console (HMC) and the HMC can populate the data structure with south-side data and then pass the structure to the hypervisor to replace the data structure on the adapter. In another embodiment the hypervisor may copy the data structure to the HMC and the HMC can instruct the hypervisor to fill-in the data structure, a virtual function at a time, with south-side management data associations. The administrator can assign south-side data, such as a MAC address for a virtual instance of an Ethernet device, to LPARs sharing the adapter. Thus, a standard way to manage the south-side data of virtual functions is provided.

PCI Function South-Side Data Management

The present invention provides a method, computer program product, and data processing system for efficiently recovering state and performing failover of a network offload engine (904). The present invention distinguishes between the hard state and the soft state of a protocol. Hard state is state information that, when lost, leads to incorrect protocol behavior. Soft state is state information that may be lost or become inconsistent without loss of correctness. The present invention ensures correctness by always being able to recover (1002) the hard state of the protocol. A preferred embodiment of the present invention performs a failover of a network offload engine (904) by temporarily blocking (908) the reception of network packets, recovering (1002) hard state from host information, resuming network operation using a substitute network offload engine (906), and recovering (1008) soft state from the subsequent network activity.

Douglas M. Freimuth

Papers

Placement of virtual machines based on server cost and network cost

Kernel Mechanisms for Service Differentiation in Overloaded Web Servers

Multi-root I/O virtualization using separate management facilities of multiple logical partitions

PCI Function South-Side Data Management

State recovery and failover of intelligent network adapters