Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial
Amal Feriani,Ekram Hossain +1 more
Reads0
Chats0
TLDR
This tutorial focuses on the role of DRL with an emphasis on deep Multi-Agent Reinforcement Learning (MARL) for AI-enabled wireless networks, and provides a selective description of RL algorithms such as Model-Based RL (MBRL) and cooperative MARL and highlights their potential applications in future wireless networks.Abstract:
Deep Reinforcement Learning (DRL) has recently witnessed significant advances that have led to multiple successes in solving sequential decision-making problems in various domains, particularly in wireless communications. The next generation of wireless networks is expected to provide scalable, low-latency, ultra-reliable services empowered by the application of data-driven Artificial Intelligence (AI). The key enabling technologies of future wireless networks, such as intelligent meta-surfaces, aerial networks, and AI at the edge, involve more than one agent which motivates the importance of multi-agent learning techniques. Furthermore, cooperation is central to establishing self-organizing, self-sustaining, and decentralized networks. In this context, this tutorial focuses on the role of DRL with an emphasis on deep Multi-Agent Reinforcement Learning (MARL) for AI-enabled wireless networks. The first part of this paper will present a clear overview of the mathematical frameworks for single-agent RL and MARL. The main idea of this work is to motivate the application of RL beyond the model-free perspective which was extensively adopted in recent years. Thus, we provide a selective description of RL algorithms such as Model-Based RL (MBRL) and cooperative MARL and we highlight their potential applications in future wireless networks. Finally, we overview the state-of-the-art of MARL in fields such as Mobile Edge Computing (MEC), Unmanned Aerial Vehicles (UAV) networks, and cell-free massive MIMO, and identify promising future research directions. We expect this tutorial to stimulate more research endeavors to build scalable and decentralized systems based on MARL.read more
Citations
More filters
Journal ArticleDOI
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges
Mingzhe Chen,Deniz Gunduz,Kaibin Huang,Walid Saad,Mehdi Bennis,Aneta Vulgarakis Feljan,H. Vincent Poor +6 more
TL;DR: In this paper, the authors provide a comprehensive study of how distributed learning can be efficiently and effectively deployed over wireless edge networks, including federated learning, federated distillation, distributed inference, and multi-agent reinforcement learning.
Journal ArticleDOI
Evolution of NOMA Toward Next Generation Multiple Access (NGMA) for 6G
TL;DR: In this article , the authors explore the evolution of NOMA with a particular focus on non-orthogonal multiple access (NOMA), and discuss several possible candidate techniques.
Journal ArticleDOI
Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications
TL;DR: The latest applications of DML in power control, spectrum management, user association, and edge cloud computing, and the potential adversarial attacks faced by DML applications are reviewed, and state-of-the-art countermeasures to preserve privacy and security are described.
Posted Content
Evolution of Non-Terrestrial Networks From 5G to 6G: A Survey.
Mohammad Mahdi Azari,Sourabh Solanki,Symeon Chatzinotas,Oltjon Kodheli,Hazem Sallouha,Achiel Colpaert,Jesus Fabian Mendoza Montoya,Sofie Pollin,Alireza Haqiqatnejad,Arsham Mostaani,Eva Lagunas,Bjorn Ottersten +11 more
TL;DR: In this article, a comprehensive survey of the evolution of non-terrestrial networks (NTNs) highlighting its relevance to 5G networks and how it will play a pivotal role in the development of 6G and beyond wireless networks is presented.
Journal ArticleDOI
Evolution of Non-Terrestrial Networks From 5G to 6G: A Survey
TL;DR: In this article , the authors comprehensively survey the evolution of non-terrestrial networks (NTNs) highlighting their relevance to 5G networks and essentially, how it will play a pivotal role in the development of 6G ecosystem.
References
More filters
Book
Reinforcement Learning: An Introduction
TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Proceedings ArticleDOI
TensorFlow: a system for large-scale machine learning
Martín Abadi,Paul Barham,Jianmin Chen,Zhifeng Chen,Andy Davis,Jeffrey Dean,Matthieu Devin,Sanjay Ghemawat,Geoffrey Irving,Michael Isard,Manjunath Kudlur,Josh Levenberg,Rajat Monga,Sherry Moore,Derek G. Murray,Benoit Steiner,Paul A. Tucker,Vijay K. Vasudevan,Pete Warden,Martin Wicke,Yuan Yu,Xiaoqiang Zheng +21 more
TL;DR: TensorFlow as mentioned in this paper is a machine learning system that operates at large scale and in heterogeneous environments, using dataflow graphs to represent computation, shared state, and the operations that mutate that state.
Proceedings Article
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Adam Paszke,Sam Gross,Francisco Massa,Adam Lerer,James Bradbury,Gregory Chanan,Trevor Killeen,Zeming Lin,Natalia Gimelshein,Luca Antiga,Alban Desmaison,Andreas Kopf,Edward Z. Yang,Zachary DeVito,Martin Raison,Alykhan Tejani,Sasank Chilamkurthy,Benoit Steiner,Lu Fang,Junjie Bai,Soumith Chintala +20 more
TL;DR: This paper details the principles that drove the implementation of PyTorch and how they are reflected in its architecture, and explains how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance.
Posted Content
Proximal Policy Optimization Algorithms
TL;DR: A new family of policy gradient methods for reinforcement learning, which alternate between sampling data through interaction with the environment, and optimizing a "surrogate" objective function using stochastic gradient ascent, are proposed.
Posted Content
Playing Atari with Deep Reinforcement Learning
Volodymyr Mnih,Koray Kavukcuoglu,David Silver,Alex Graves,Ioannis Antonoglou,Daan Wierstra,Martin Riedmiller +6 more
TL;DR: This work presents the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning, which outperforms all previous approaches on six of the games and surpasses a human expert on three of them.