Open AccessPosted Content
Reinforcement Learning for Adaptive Caching with Dynamic Storage Pricing
TLDR
In this article, a generic time-varying fetching and caching costs are formulated to minimize the aggregate cost across files and time, and a light-weight online solver for the corresponding optimization is employed to find optimal fetch-cache decisions.Abstract:
Small base stations (SBs) of fifth-generation (5G) cellular networks are envisioned to have storage devices to locally serve requests for reusable and popular contents by \emph{caching} them at the edge of the network, close to the end users. The ultimate goal is to shift part of the predictable load on the back-haul links, from on-peak to off-peak periods, contributing to a better overall network performance and service experience. To enable the SBs with efficient \textit{fetch-cache} decision-making schemes operating in dynamic settings, this paper introduces simple but flexible generic time-varying fetching and caching costs, which are then used to formulate a constrained minimization of the aggregate cost across files and time. Since caching decisions per time slot influence the content availability in future slots, the novel formulation for optimal fetch-cache decisions falls into the class of dynamic programming. Under this generic formulation, first by considering stationary distributions for the costs and file popularities, an efficient reinforcement learning-based solver known as value iteration algorithm can be used to solve the emerging optimization problem. Later, it is shown that practical limitations on cache capacity can be handled using a particular instance of the generic dynamic pricing formulation. Under this setting, to provide a light-weight online solver for the corresponding optimization, the well-known reinforcement learning algorithm, $Q$-learning, is employed to find optimal fetch-cache decisions. Numerical tests corroborating the merits of the proposed approach wrap up the paper.read more
Citations
More filters
Journal ArticleDOI
Deep Reinforcement Learning for Adaptive Caching in Hierarchical Content Delivery Networks
TL;DR: To model the two-way interactive influence between caching decisions at the parent and leaf nodes, a reinforcement learning (RL) framework is put forth and a scalable deep RL approach is pursued, which relies on a hyper-deep Q-network to learn the Q-function, and thus the optimal caching policy, in an online fashion.
Journal ArticleDOI
Applying machine learning techniques for caching in next-generation edge networks: A comprehensive survey
TL;DR: A comprehensive taxonomy of machine learning techniques for in-network caching in edge networks is formulated and a comparative analysis of the state-of-the-art literature is presented with respect to the parameters identified in the taxonomy.
Journal ArticleDOI
Adaptive Edge Association for Wireless Digital Twin Networks in 6G
TL;DR: A wireless digital twin edge network model is proposed by integrating digital twin with edge networks to enable new functionalities, such as hyper-connected experience and low-latency edge computing, and reduced system cost and enhanced convergence rate for dynamic network states.
Journal ArticleDOI
Artificial Intelligence for Wireless Caching: Schemes, Performance, and Challenges
TL;DR: A systematic survey of state-of-the-art intelligent data caching approaches based on learning mechanism to optimize data caching based on accurate predictions of users’ data requests and data popularity profile is provided.
Journal ArticleDOI
Reinforcement Learning for Efficient and Fair Coexistence Between LTE-LAA and Wi-Fi
TL;DR: An analytical model is developed to evaluate the throughput performance of Category 4 (Cat 4) algorithm agreed in 3GPP release 13 and it is shown that both proposed learning algorithms can significantly improve the total throughput performance while satisfying the fairness constraints.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Book
Convex Optimization
Stephen Boyd,Lieven Vandenberghe +1 more
TL;DR: In this article, the focus is on recognizing convex optimization problems and then finding the most appropriate technique for solving them, and a comprehensive introduction to the subject is given. But the focus of this book is not on the optimization problem itself, but on the problem of finding the appropriate technique to solve it.
Journal ArticleDOI
Technical Note : \cal Q -Learning
Chris Watkins,Peter Dayan +1 more
TL;DR: This paper presents and proves in detail a convergence theorem forQ-learning based on that outlined in Watkins (1989), showing that Q-learning converges to the optimum action-values with probability 1 so long as all actions are repeatedly sampled in all states and the action- values are represented discretely.
Proceedings ArticleDOI
Web caching and Zipf-like distributions: evidence and implications
TL;DR: This paper investigates the page request distribution seen by Web proxy caches using traces from a variety of sources and considers a simple model where the Web accesses are independent and the reference probability of the documents follows a Zipf-like distribution, suggesting that the various observed properties of hit-ratios and temporal locality are indeed inherent to Web accesse observed by proxies.