Multi-class latency-bounded Web services

TLDR: The goal of this work is to design a general "front-end" algorithm that uses a general service abstraction to adaptively control not only the latency of a particular class, but also to assess the inter-class relationships.

Abstract: Two recent advances have resulted in significant improvements in Web server quality of service. First, both centralized and distributed Web servers can provide isolation among service classes by fairly distributing system resources. Second, session admission control can protect classes from performance degradation due to overload. The goal of this work is to design a general "front-end" algorithm that uses these two building blocks to support a new Web service model, namely, multi-class services which control response latencies to within pre-specified targets. Our key technique is to devise a general service abstraction to adaptively control not only the latency of a particular class, but also to assess the inter-class relationships. In this way, we capture the extent to which classes are isolated or share system resources (as determined by the server architecture and system internals) and hence their effects on each other's QoS. For example, if the server provides class isolation (i.e., a minimum fraction of system resources independent of other classes), yet also allows a class to utilize unused resources from other classes, the algorithm infers and exploits this behavior without an explicit low level model of the server. Thus, as new functionalities are incorporated into Web servers, the approach naturally exploits their properties to efficiently satisfy the classes' performance targets. We validate the scheme with trace driven simulations.