Author
O. Bahat
Bio: O. Bahat is an academic researcher from University of Maryland, College Park. The author has contributed to research in topics: Cache & Markov process. The author has an hindex of 1, co-authored 1 publications receiving 43 citations.
Topics: Cache, Markov process, Locality of reference
Papers
More filters
09 Jul 2003
TL;DR: This paper shows that a simple Markov stationary replacement policy, called the policy C*/sub 0/, minimizes the long-run average metric induced by nonuniform document costs when document eviction is optional, and proposes a framework for operating caching systems with multiple performance metrics by solving a constrained caching problem with a single constraint.
Abstract: Replacement policies for general caching applications and Web caching in particular have been discussed extensively in the literature. Many ad-hoc policies have been proposed that attempt to take adavantage of the retrieval latency of documents, their size, the popularity of references and temporal locality of requested documents. However, the problem of finding optimal replacement policies under these factors has not been pursued in any systematic manner. In this paper, we take a step in that direction: we first show, still under the independent reference model, that a simple Markov stationary replacement policy, called the policy C*/sub 0/, minimizes the long-run average metric induced by nonuniform document costs when document eviction is optional. We then propose a framework for operating caching systems with multiple performance metrics. We do so by solving a constrained caching problem with a single constraint. The resulting constrained optimal replacement policy is obtained by simple randomization between two Markov stationary optimal replacement policies C*/sub 0/ but induced by different costs.
43 citations
Cited by
More filters
01 Feb 2017
TL;DR: These results show that formalizing cache replacement yields practical benefits, and propose that practical policies should replace lines based on their economic value added (EVA), the difference of their expected hits from the average.
Abstract: Much prior work has studied cache replacement, but a large gap remains between theory and practice. The design of many practical policies is guided by the optimal policy, Belady's MIN. However, MIN assumes perfect knowledge of the future that is unavailable in practice, and the obvious generalizationsof MIN are suboptimal with imperfect information. What, then, is the right metric for practical cache replacement?We propose that practical policies should replace lines based on their economic value added (EVA), the difference of their expected hits from the average. Drawing on the theory of Markov decision processes, we discuss why this metric maximizes the cache's hit rate. We present an inexpensive implementation ofEVA and evaluate it exhaustively. EVA outperforms several prior policies and saves area at iso-performance. These results show that formalizing cache replacement yields practical benefits.
55 citations
TL;DR: This work shows that evicting more than the minimum number of code blocks from the code cache results in less run-time overhead than the existing alternatives, and describes and evaluates a generational approach to code cache management that makes it easy to identify long-lived code blocks and simultaneously avoid any fragmentation because of the eviction of short-lived blocks.
Abstract: Dynamic binary optimizers store altered copies of original program instructions in software-managed code caches in order to maximize reuse of transformed code. Code caches store code blocks that may vary in size, reference other code blocks, and carry a high replacement overhead. These unique constraints reduce the effectiveness of conventional cache management policies. Our work directly addresses these unique constraints and presents several contributions to the code-cache management problem. First, we show that evicting more than the minimum number of code blocks from the code cache results in less run-time overhead than the existing alternatives. Such granular evictions reduce overall execution time, as the fixed costs of invoking the eviction mechanism are amortized across multiple cache insertions. Second, a study of the ideal lifetimes of dynamically generated code blocks illustrates the benefit of a replacement algorithm based on a generational heuristic. We describe and evaluate a generational approach to code cache management that makes it easy to identify long-lived code blocks and simultaneously avoid any fragmentation because of the eviction of short-lived blocks. Finally, we present results from an implementation of our generational approach in the DynamoRIO framework and illustrate that, as dynamic optimization systems become more prevalent, effective code cache-management policies will be essential for reliable, scalable performance of modern applications.
40 citations
12 Jun 2018
TL;DR: This analysis shows that current caching systems are in fact still far from optimal, suffering 11-43% more cache misses than OPT, whereas the best prior offline bounds suggest that there is essentially no room for improvement.
Abstract: Many recent caching systems aim to improve miss ratios, but there is no good sense among practitioners of how much further miss ratios can be improved. In other words, should the systems community continue working on this problem? Currently, there is no principled answer to this question. In practice, object sizes often vary by several orders of magnitude, where computing the optimal miss ratio (OPT) is known to be NP-hard. The few known results on caching with variable object sizes provide very weak bounds and are impractical to compute on traces of realistic length. We propose a new method to compute upper and lower bounds on OPT. Our key insight is to represent caching as a min-cost flow problem, hence we call our method the flow-based offline optimal (FOO). We prove that, under simple independence assumptions, FOO's bounds become tight as the number of objects goes to infinity. Indeed, FOO's error over 10M requests of production CDN and storage traces is negligible: at most 0.3%. FOO thus reveals, for the first time, the limits of caching with variable object sizes. While FOO is very accurate, it is computationally impractical on traces with hundreds of millions of requests. We therefore extend FOO to obtain more efficient bounds on OPT, which we call practical flow-based offline optimal (PFOO). We evaluate PFOO on several full production traces and use it to compare OPT to prior online policies. This analysis shows that current caching systems are in fact still far from optimal, suffering 11-43% more cache misses than OPT, whereas the best prior offline bounds suggest that there is essentially no room for improvement.
35 citations
TL;DR: This analysis shows that current caching systems are in fact still far from optimal, suffering 11--43% more cache misses than OPT, whereas the best prior offline bounds suggest that there is essentially no room for improvement.
Abstract: Many recent caching systems aim to improve hit ratios, but there is no good sense among practitioners of how much further hit ratios can be improved. In other words, should the systems community continue working on this problem? Currently, there is no principled answer to this question. Most prior work assumes that objects have the same size, but in practice object sizes often vary by several orders of magnitude. The few known results for variable object sizes provide very weak guarantees and are impractical to compute on traces of realistic length. We propose a new method to compute the offline optimal hit ratio under variable object sizes. Our key insight is to represent caching as a min-cost flow problem, hence we call our method the flow-based offline optimal (FOO). We show that, under simple independence assumptions and Zipf popularities, FOO's bounds become tight as the number of objects goes to infinity. From FOO we develop fast, practical methods to compute nearly tight bounds for the optimal hit ratio, which we call practical flow-based offline optimal (P-FOO). P-FOO enables the first analysis of optimal caching on realistic traces with hundreds of millions of requests. We evaluate P-FOO on several production traces, where results show that recent caching systems are still far from optimal.
35 citations
10 Jun 2014
TL;DR: This work designs and implements a new memory partitioning scheme, called PSA, which replaces that of vanilla Memcached, and shows that PSA is capable of “adapting” to the dynamics of clients' requests and object size distributions, thus defeating the calcification problem.
Abstract: Memcached is a popular component of modern Web architectures, which allows fast response times - a fundamental performance index for measuring the Quality of Experience of end-users - for serving popular objects. In this work, we study how memory partitioning in Memcached works and how it affects system performance in terms of hit rate. Memcached divides the memory into different classes proportionally to the percentage of requests for objects of different sizes. Once all the available memory has been allocated, reallocation is not possible or limited, a problem called “calcification”. Calcification constitutes a symptom indicating that current memory partitioning mechanisms require a more careful design. Using an experimental approach, we show the negative impact of calcification on an important performance metric, the hit rate. We then proceed to design and implement a new memory partitioning scheme, called PSA, which replaces that of vanilla Memcached. With PSA, Memcached achieves a higher hit rate than what is obtained with the default memory partitioning mechanism, even in the absence of calcification. Moreover, we show that PSA is capable of “adapting” to the dynamics of clients' requests and object size distributions, thus defeating the calcification problem.
34 citations