Showing papers on "Smart Cache published in 2021"

Jointly Optimize Capacity, Latency and Engagement in Large-scale Recommendation Systems

Abstract: As the recommendation systems behind commercial services scale up and apply more and more sophisticated machine learning models, it becomes important to optimize computational cost (capacity) and runtime latency, besides the traditional objective of user engagement. Caching recommended results and reusing them later is a common technique used to reduce capacity and latency. However, the standard caching approach negatively impacts user engagement. To overcome the challenge, this paper presents an approach to optimizing capacity, latency and engagement simultaneously. We propose a smart caching system including a lightweight adjuster model to refresh the cached ranking scores, achieving significant capacity savings without impacting ranking quality. To further optimize latency, we introduce a prefetching strategy which leverages the smart cache. Our production deployment on Facebook Marketplace demonstrates that the approach reduces capacity demand by 50% and p75 end-to-end latency by 35%. While Facebook Marketplace is used as a case study, the approach is applicable to other industrial recommendation systems as well.

Effective Cache Apportioning for Performance Isolation Under Compiler Guidance.

Abstract: With a growing number of cores per socket in modern data-centers where multi-tenancy of a diverse set of applications must be efficiently supported, effective sharing of the last level cache is a very important problem. This is challenging because modern workloads exhibit dynamic phase behavior - their cache requirements & sensitivity vary across different execution points. To tackle this problem, we propose Com-CAS, a compiler-guided cache apportioning system that provides smart cache allocation to co-executing applications in a system. The front-end of Com-CAS is primarily a compiler-framework equipped with learning mechanisms to predict cache requirements, while the backend consists of an allocation framework with a pro-active scheduler that apportions cache dynamically to co-executing applications. Our system improved average throughput by 21%, with a maximum of 54% while maintaining the worst individual application execution time degradation within 15% to meet SLA requirements.

Abusing Cache Line Dirty States to Leak Information in Commercial Processors.

Abstract: Caches have been used to construct various types of covert and side channels to leak information. Most of the previous cache channels exploit the timing difference between cache hits and cache misses. However, we introduce a new and broader classification of cache covert channel attacks: Hit+Miss, Hit+Hit, Miss+Miss. We highlight that cache misses (or cache hits) in different states may have more significant time differences, which can be used as timing channels. Based on the classification, We propose a new type of stable and stealthy Miss+Miss cache channel. The write-back caches are widely deployed in modern processors. This paper presents in detail how to use replacement latency difference to construct timing-based channels (calles WB channel) to leak information in the write-back cache: any modification to a cache line by a sender will set the cache line to the dirty state, and the receiver can observe this through measuring the latency to replace this cache set. We also demonstrate how senders could exploit a different number of dirty cache lines in a cache set to improve transmission bandwidth with symbols encoding multiple bits. The peak transmission bandwidths of the WB channels in commercial systems can vary between 1300 to 4400 Kbps per cache set in the hyper-threaded setting without shared memory between the sender and the receiver. Different from most existing cache channels that always target specific memory addresses, the new WB channels focus on the cache set and cache line states, making the channel hard to be disturbed by other processes on the core and can still work in the cache using a random replacement policy. We also analyzed the stealthiness of WB channels from the perspective of the number of cache loads and cache miss rates. Further, This paper discusses and evaluates possible defenses. The paper finishes by discussing various forms of side-channel attacks.

Markov transition and smart cache congestion control for IoT enabled wireless mesh networks

Abstract: Wireless Mesh Networking (WMN) is the latest Internet framework that provides a comprehensive range for Subsequent Internet (SI) prototype. Despite significant advantages provided by WMN, its practical distribution to connect Internet of Things (IoT) networks caused exorbitant congestion and restricted bandwidth. Motivated by this, a novel mechanism that ensures control in the manifestation of mobbing, end-to-end delay, energy consumption for enhancing the network performance of IoT-enabled WMN is presented. The proposed method is called as an Integrated Markov State Transition and Open Loop Smart Caching (MST-OLSC) for congestion control in IoT-enabled WMN. The proposed method uses Markov state transition scheduling model to differentiate the states of the incoming data packets from the host computer. This is performed by applying the State Betweenness centrality. Next, Congestion Control Token Caching mechanism is applied with the objective of controlling the congestion by means of caching via overflow with well-balanced isolation between regulated and unregulated flow of data packet. Finally, Open Loop Smart Caching is presented to ensure constant data rate, thereby providing fair inflow and outflow between the incoming and outgoing data packets. The evaluation results of MST-OLSC ensure higher network performance with minimum end-to-end delay, energy consumption and higher packet delivery rate is achieved with respect to inflated IoT nodes in WMN.