A Tutorial on Ultrareliable and Low-Latency Communications in 6G: Integrating Domain Knowledge Into Deep Learning
04 Mar 2021-Vol. 109, Iss: 3, pp 204-246
TL;DR: In this paper, the authors discuss the potential of applying supervised/unsupervised deep learning and deep reinforcement learning in ultrareliable and low-latency communications (URLLCs) in future 6G networks.
Abstract: As one of the key communication scenarios in the fifth-generation and also the sixth-generation (6G) mobile communication networks, ultrareliable and low-latency communications (URLLCs) will be central for the development of various emerging mission-critical applications. State-of-the-art mobile communication systems do not fulfill the end-to-end delay and overall reliability requirements of URLLCs. In particular, a holistic framework that takes into account latency, reliability, availability, scalability, and decision-making under uncertainty is lacking. Driven by recent breakthroughs in deep neural networks, deep learning algorithms have been considered as promising ways of developing enabling technologies for URLLCs in future 6G networks. This tutorial illustrates how domain knowledge (models, analytical tools, and optimization frameworks) of communications and networking can be integrated into different kinds of deep learning algorithms for URLLCs. We first provide some background of URLLCs and review promising network architectures and deep learning frameworks for 6G. To better illustrate how to improve learning algorithms with domain knowledge, we revisit model-based analytical tools and cross-layer optimization frameworks for URLLCs. Following this, we examine the potential of applying supervised/unsupervised deep learning and deep reinforcement learning in URLLCs and summarize related open problems. Finally, we provide simulation and experimental results to validate the effectiveness of different learning algorithms and discuss future directions.
Citations
More filters
TL;DR: This tutorial focuses on the role of DRL with an emphasis on deep Multi-Agent Reinforcement Learning (MARL) for AI-enabled wireless networks, and provides a selective description of RL algorithms such as Model-Based RL (MBRL) and cooperative MARL and highlights their potential applications in future wireless networks.
Abstract: Deep Reinforcement Learning (DRL) has recently witnessed significant advances that have led to multiple successes in solving sequential decision-making problems in various domains, particularly in wireless communications. The next generation of wireless networks is expected to provide scalable, low-latency, ultra-reliable services empowered by the application of data-driven Artificial Intelligence (AI). The key enabling technologies of future wireless networks, such as intelligent meta-surfaces, aerial networks, and AI at the edge, involve more than one agent which motivates the importance of multi-agent learning techniques. Furthermore, cooperation is central to establishing self-organizing, self-sustaining, and decentralized networks. In this context, this tutorial focuses on the role of DRL with an emphasis on deep Multi-Agent Reinforcement Learning (MARL) for AI-enabled wireless networks. The first part of this paper will present a clear overview of the mathematical frameworks for single-agent RL and MARL. The main idea of this work is to motivate the application of RL beyond the model-free perspective which was extensively adopted in recent years. Thus, we provide a selective description of RL algorithms such as Model-Based RL (MBRL) and cooperative MARL and we highlight their potential applications in future wireless networks. Finally, we overview the state-of-the-art of MARL in fields such as Mobile Edge Computing (MEC), Unmanned Aerial Vehicles (UAV) networks, and cell-free massive MIMO, and identify promising future research directions. We expect this tutorial to stimulate more research endeavors to build scalable and decentralized systems based on MARL.
135 citations
TL;DR: In this article , the authors make a beneficial effort to explore the role of AI, including machine learning algorithms and deep learning architectures, in the foundation and development of the metaverse, and convey a comprehensive investigation of AI-based methods concerning several technical aspects (e.g., natural language processing, machine vision, blockchain, networking, digital twin, and neural interface).
114 citations
TL;DR: In this paper , the authors provide a vision for scalable and trustworthy edge AI systems with integrated design of wireless communication strategies and decentralized machine learning models, as well as a holistic end-to-end system architecture to support edge AI.
Abstract: The thriving of artificial intelligence (AI) applications is driving the further evolution of wireless networks. It has been envisioned that 6G will be transformative and will revolutionize the evolution of wireless from “connected things” to “connected intelligence”. However, state-of-the-art deep learning and big data analytics based AI systems require tremendous computation and communication resources, causing significant latency, energy consumption, network congestion, and privacy leakage in both of the training and inference processes. By embedding model training and inference capabilities into the network edge, edge AI stands out as a disruptive technology for 6G to seamlessly integrate sensing, communication, computation, and intelligence, thereby improving the efficiency, effectiveness, privacy, and security of 6G networks. In this paper, we shall provide our vision for scalable and trustworthy edge AI systems with integrated design of wireless communication strategies and decentralized machine learning models. New design principles of wireless networks, service-driven resource allocation optimization methods, as well as a holistic end-to-end system architecture to support edge AI will be described. Standardization, software and hardware platforms, and application scenarios are also discussed to facilitate the industrialization and commercialization of edge AI systems.
92 citations
TL;DR: In this paper, a detailed overview of the URLLC features from 5G Release 15 to Release 16 by describing how these features allow meeting ULLLC target requirements in 5G networks is presented.
Abstract: Ultra-reliable low-latency communication (URLLC) has been introduced in 5G new radio for new applications that have strict reliability and latency requirements such as augmented/virtual reality, industrial automation and autonomous vehicles. The first full set of the physical layer design of 5G release, Release 15, was finalized in December 2017. It provided a foundation for URLLC with new features such as flexible sub-carrier spacing, a sub-slot-based transmission scheme, new channel quality indicator, new modulation and coding scheme tables, and configured-grant transmission with automatic repetitions. The second 5G release, Release 16, was finalized in December 2019 and allows achieving improved metrics for latency and reliability to support new use cases of URLLC. A number of new features such as enhanced physical downlink (DL) control channel monitoring capability, new DL control information format, sub-slot physical uplink (UL) control channel transmission, sub-slot-based physical UL shared channel repetition, enhanced mobile broadband and URLLC inter-user-equipment multiplexing with cancellation indication and enhanced power control were standardized. This article provides a detailed overview of the URLLC features from 5G Release 15 to Release 16 by describing how these features allow meeting URLLC target requirements in 5G networks. The ongoing Release 17 targets further enhanced URLLC operation by improving mechanisms such as feedback, intra-user-equipment multiplexing and prioritization of traffic with different priority, support of time synchronization and new quality of service related parameters. In addition, a fundamental feature targeted in URLLC Release 17 is to enable URLLC operation over shared unlicensed spectrum. The potential directions of URLLC research in unlicensed spectrum in Release 17 are presented to serve as a bridge from URLLC in licensed spectrum in Release 16 to URLLC in unlicensed spectrum in Release 17.
88 citations
TL;DR: In this paper, the authors investigated a UAV-enabled MEC network with the consideration of multiple tasks either for computing or caching, and aimed to minimize the total energy consumption of IoT devices by jointly optimizing trajectory, communication and computing resource allocation at UAV, and task offloading decision at IoT devices.
Abstract: Recently, unmanned aerial vehicle (UAV) acts as the aerial mobile edge computing (MEC) node to help the battery-limited Internet of Things (IoT) devices relieve burdens from computation and data collection, and prolong the lifetime of operating. However, IoT devices can ONLY ask UAV for either computing or caching help, and collaborative offloading services of UAV are rarely mentioned in the literature. Moreover, IoT device has multiple mutually independent tasks, which make collaborative offloading policy design even more challenging. Therefore, we investigate a UAV-enabled MEC networks with the consideration of multiple tasks either for computing or caching. Taking the quality of experience (QoE) requirement of time-sensitive tasks into consideration, we aim to minimize the total energy consumption of IoT devices by jointly optimizing trajectory, communication and computing resource allocation at UAV, and task offloading decision at IoT devices. Since this problem has highly non-convex objective function and constraints, we first decompose the original problem into three subproblems named as trajectory optimization (
$$\mathbf {P}_{\mathbf {T}}$$
), resource allocation at UAV (
$$\mathbf {P}_{\mathbf {R}}$$
) and offloading decisions at IoT devices (
$$\mathbf {P}_{\mathbf {O}}$$
) and then propose an iterative algorithm based on block coordinate descent method to cope with them in a sequence. Numerical results demonstrate that collaborative offloading can effectively reduce IoT devices’ energy consumption while meeting different kinds of offloading services, and satisfy the QoE requirement of time-sensitive tasks at IoT devices.
63 citations
References
More filters
TL;DR: This final installment of the paper considers the case where the signals or the messages or both are continuously variable, in contrast with the discrete nature assumed until now.
Abstract: In this final installment of the paper we consider the case where the signals or the messages or both are continuously variable, in contrast with the discrete nature assumed until now. To a considerable extent the continuous case can be obtained through a limiting process from the discrete case by dividing the continuum of messages and signals into a large but finite number of small regions and calculating the various parameters involved on a discrete basis. As the size of the regions is decreased these parameters in general approach as limits the proper values for the continuous case. There are, however, a few new effects that appear and also a general change of emphasis in the direction of specialization of the general results to particular cases.
65,425 citations
[...]
TL;DR: Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years, and will have many more successes in the near future because it requires very little engineering by hand and can easily take advantage of increases in the amount of available computation and data.
Abstract: Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.
46,982 citations
Book•
01 Jan 1988TL;DR: This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Abstract: Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.
37,989 citations
TL;DR: Back-propagation repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector, which helps to represent important features of the task domain.
Abstract: We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal ‘hidden’ units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured by the interactions of these units. The ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure1.
23,814 citations
TL;DR: This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.
Abstract: The theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.
23,074 citations