scispace - formally typeset
Open accessJournal ArticleDOI: 10.1109/JPROC.2021.3053601

A Tutorial on Ultrareliable and Low-Latency Communications in 6G: Integrating Domain Knowledge Into Deep Learning

04 Mar 2021-Vol. 109, Iss: 3, pp 204-246
Abstract: As one of the key communication scenarios in the fifth-generation and also the sixth-generation (6G) mobile communication networks, ultrareliable and low-latency communications (URLLCs) will be central for the development of various emerging mission-critical applications. State-of-the-art mobile communication systems do not fulfill the end-to-end delay and overall reliability requirements of URLLCs. In particular, a holistic framework that takes into account latency, reliability, availability, scalability, and decision-making under uncertainty is lacking. Driven by recent breakthroughs in deep neural networks, deep learning algorithms have been considered as promising ways of developing enabling technologies for URLLCs in future 6G networks. This tutorial illustrates how domain knowledge (models, analytical tools, and optimization frameworks) of communications and networking can be integrated into different kinds of deep learning algorithms for URLLCs. We first provide some background of URLLCs and review promising network architectures and deep learning frameworks for 6G. To better illustrate how to improve learning algorithms with domain knowledge, we revisit model-based analytical tools and cross-layer optimization frameworks for URLLCs. Following this, we examine the potential of applying supervised/unsupervised deep learning and deep reinforcement learning in URLLCs and summarize related open problems. Finally, we provide simulation and experimental results to validate the effectiveness of different learning algorithms and discuss future directions.

... read more

Topics: Unsupervised learning (60%), Reinforcement learning (57%), Deep learning (56%) ... show more
Citations
  More

37 results found


Open accessJournal ArticleDOI: 10.1109/COMST.2021.3063822
Amal Feriani1, Ekram Hossain1Institutions (1)
Abstract: Deep Reinforcement Learning (DRL) has recently witnessed significant advances that have led to multiple successes in solving sequential decision-making problems in various domains, particularly in wireless communications. The next generation of wireless networks is expected to provide scalable, low-latency, ultra-reliable services empowered by the application of data-driven Artificial Intelligence (AI). The key enabling technologies of future wireless networks, such as intelligent meta-surfaces, aerial networks, and AI at the edge, involve more than one agent which motivates the importance of multi-agent learning techniques. Furthermore, cooperation is central to establishing self-organizing, self-sustaining, and decentralized networks. In this context, this tutorial focuses on the role of DRL with an emphasis on deep Multi-Agent Reinforcement Learning (MARL) for AI-enabled wireless networks. The first part of this paper will present a clear overview of the mathematical frameworks for single-agent RL and MARL. The main idea of this work is to motivate the application of RL beyond the model-free perspective which was extensively adopted in recent years. Thus, we provide a selective description of RL algorithms such as Model-Based RL (MBRL) and cooperative MARL and we highlight their potential applications in future wireless networks. Finally, we overview the state-of-the-art of MARL in fields such as Mobile Edge Computing (MEC), Unmanned Aerial Vehicles (UAV) networks, and cell-free massive MIMO, and identify promising future research directions. We expect this tutorial to stimulate more research endeavors to build scalable and decentralized systems based on MARL.

... read more

15 Citations


Open accessJournal ArticleDOI: 10.1186/S13638-020-01861-8
Wen-Tao Li1, Mingxiong Zhao1, Yu-Hui Wu1, Jun-Jie Yu1  +3 moreInstitutions (1)
Abstract: Recently, unmanned aerial vehicle (UAV) acts as the aerial mobile edge computing (MEC) node to help the battery-limited Internet of Things (IoT) devices relieve burdens from computation and data collection, and prolong the lifetime of operating. However, IoT devices can ONLY ask UAV for either computing or caching help, and collaborative offloading services of UAV are rarely mentioned in the literature. Moreover, IoT device has multiple mutually independent tasks, which make collaborative offloading policy design even more challenging. Therefore, we investigate a UAV-enabled MEC networks with the consideration of multiple tasks either for computing or caching. Taking the quality of experience (QoE) requirement of time-sensitive tasks into consideration, we aim to minimize the total energy consumption of IoT devices by jointly optimizing trajectory, communication and computing resource allocation at UAV, and task offloading decision at IoT devices. Since this problem has highly non-convex objective function and constraints, we first decompose the original problem into three subproblems named as trajectory optimization ( $$\mathbf {P}_{\mathbf {T}}$$ ), resource allocation at UAV ( $$\mathbf {P}_{\mathbf {R}}$$ ) and offloading decisions at IoT devices ( $$\mathbf {P}_{\mathbf {O}}$$ ) and then propose an iterative algorithm based on block coordinate descent method to cope with them in a sequence. Numerical results demonstrate that collaborative offloading can effectively reduce IoT devices’ energy consumption while meeting different kinds of offloading services, and satisfy the QoE requirement of time-sensitive tasks at IoT devices.

... read more

Topics: Mobile edge computing (55%)

13 Citations


Open accessJournal ArticleDOI: 10.1109/ACCESS.2020.3046773
01 Jan 2021-IEEE Access
Abstract: Ultra-reliable low-latency communication (URLLC) has been introduced in 5G new radio for new applications that have strict reliability and latency requirements such as augmented/virtual reality, industrial automation and autonomous vehicles. The first full set of the physical layer design of 5G release, Release 15, was finalized in December 2017. It provided a foundation for URLLC with new features such as flexible sub-carrier spacing, a sub-slot-based transmission scheme, new channel quality indicator, new modulation and coding scheme tables, and configured-grant transmission with automatic repetitions. The second 5G release, Release 16, was finalized in December 2019 and allows achieving improved metrics for latency and reliability to support new use cases of URLLC. A number of new features such as enhanced physical downlink (DL) control channel monitoring capability, new DL control information format, sub-slot physical uplink (UL) control channel transmission, sub-slot-based physical UL shared channel repetition, enhanced mobile broadband and URLLC inter-user-equipment multiplexing with cancellation indication and enhanced power control were standardized. This article provides a detailed overview of the URLLC features from 5G Release 15 to Release 16 by describing how these features allow meeting URLLC target requirements in 5G networks. The ongoing Release 17 targets further enhanced URLLC operation by improving mechanisms such as feedback, intra-user-equipment multiplexing and prioritization of traffic with different priority, support of time synchronization and new quality of service related parameters. In addition, a fundamental feature targeted in URLLC Release 17 is to enable URLLC operation over shared unlicensed spectrum. The potential directions of URLLC research in unlicensed spectrum in Release 17 are presented to serve as a bridge from URLLC in licensed spectrum in Release 16 to URLLC in unlicensed spectrum in Release 17.

... read more

Topics: Spectrum management (56%), Control channel (54%), Transmission (telecommunications) (54%) ... show more

11 Citations


Open accessPosted Content
Abstract: Densification of network base stations is indispensable to achieve the stringent Quality of Service (QoS) requirements of future mobile networks. However, with a dense deployment of transmitters, interference management becomes an arduous task. To solve this issue, exploring radically new network architectures with intelligent coordination and cooperation capabilities is crucial. This survey paper investigates the emerging user-centric cell-free massive multiple-input multiple-output (MIMO) network architecture that sets a foundation for future mobile networks. Such networks use a dense deployment of distributed units (DUs) to serve users; the crucial difference from the traditional cellular paradigm is that a specific serving cluster of DUs is defined for each user. This framework provides macro diversity, power efficiency, interference management, and robust connectivity. Most importantly, the user-centric approach eliminates cell edges, thus contributing to uniform coverage and performance for users across the network area. We present here a guide to the key challenges facing the deployment of this network scheme and contemplate the solutions being proposed for the main bottlenecks facing cell-free communications. Specifically, we survey the literature targeting the fronthaul, then we scan the details of the channel estimation required, resource allocation, delay, and scalability issues. Furthermore, we highlight some technologies that can provide a management platform for this scheme such as distributed software-defined network (SDN) and self-organizing network (SON). Our article serves as a check point that delineates the current status and indicates future directions for this area in a comprehensive manner.

... read more

Topics: Network architecture (60%), Quality of service (54%), Scalability (53%) ... show more

5 Citations


Open accessPosted ContentDOI: 10.1109/JIOT.2021.3107242
Jie Ding1, Mahyar Nemati1, Shiva Raj Pokhrel1, Ok-Sun Park2  +2 moreInstitutions (3)
Abstract: Enabling ultra-reliable low-latency communication (URLLC) with stringent requirements for transmitting data packets (e.g., 99.999% reliability and 1 millisecond latency) presents considerable challenges in uplink transmissions. For each packet transmission over dynamically allocated network radio resources, the conventional random access protocols are based on a request-grant scheme. This induces excessive latency and necessitates reliable control signalling, resulting overhead. To address these problems, grant-free (GF) solutions are proposed in the fifth-generation (5G) new radio (NR). In this paper, an overview and vision of the state-of-the-art in enabling GF URLLC are presented. In particular, we first provide a comprehensive review of NR specifications and techniques for URLLC, discuss underlying principles, and highlight impeding issues of enabling GF URLLC. Furthermore, we briefly explain two key phenomena of massive multiple-input multiple-output (mMIMO) (i.e., channel hardening and favorable propagation) and build several deep insights into how celebrated mMIMO features can be exploited to enhance the performance of GF URLLC. Moving further ahead, we examine the potential of cell-free (CF) mMIMO and analyze its distinctive features and benefits over mMIMO to resolve GF URLLC issues. Finally, we identify future research directions and challenges in enabling GF URLLC with CF mMIMO.A new version of the paper has been updated on 21/08/2021

... read more

5 Citations


References
  More

272 results found


Journal ArticleDOI: 10.1002/J.1538-7305.1948.TB01338.X
Abstract: In this final installment of the paper we consider the case where the signals or the messages or both are continuously variable, in contrast with the discrete nature assumed until now. To a considerable extent the continuous case can be obtained through a limiting process from the discrete case by dividing the continuum of messages and signals into a large but finite number of small regions and calculating the various parameters involved on a discrete basis. As the size of the regions is decreased these parameters in general approach as limits the proper values for the continuous case. There are, however, a few new effects that appear and also a general change of emphasis in the direction of specialization of the general results to particular cases.

... read more

60,029 Citations


Journal ArticleDOI: 10.1038/NATURE14539
Yann LeCun1, Yann LeCun2, Yoshua Bengio3, Geoffrey E. Hinton4  +1 moreInstitutions (5)
28 May 2015-Nature
Abstract: Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.

... read more

33,931 Citations


Open accessBook
Richard S. Sutton1, Andrew G. BartoInstitutions (1)
01 Jan 1988-
Abstract: Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

... read more

Topics: Learning classifier system (69%), Reinforcement learning (69%), Apprenticeship learning (65%) ... show more

32,257 Citations


Journal ArticleDOI: 10.1038/323533A0
01 Jan 1988-Nature
Abstract: We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal ‘hidden’ units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured by the interactions of these units. The ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure1.

... read more

19,542 Citations


Journal ArticleDOI: 10.1038/NATURE14236
Volodymyr Mnih1, Koray Kavukcuoglu1, David Silver1, Andrei Rusu1  +15 moreInstitutions (1)
26 Feb 2015-Nature
Abstract: The theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.

... read more

Topics: Reinforcement learning (65%), Q-learning (61%), Temporal difference learning (57%) ... show more

15,690 Citations


Performance
Metrics
No. of citations received by the Paper in previous years
YearCitations
202132
20205