Interference Management for Cellular-Connected UAVs: A Deep Reinforcement Learning Approach
TLDR
The proposed deep reinforcement learning algorithm, based on echo state network (ESN) cells, achieves better wireless latency per UAV and rate per ground user (UE) while requiring a number of steps that are comparable to a heuristic baseline that considers moving via the shortest distance toward the corresponding destinations.Abstract:
In this paper, an interference-aware path planning scheme for a network of cellular-connected unmanned aerial vehicles (UAVs) is proposed. In particular, each UAV aims at achieving a tradeoff between maximizing energy efficiency and minimizing both wireless latency and the interference caused on the ground network along its path. The problem is cast as a dynamic game among UAVs. To solve this game, a deep reinforcement learning algorithm, based on echo state network (ESN) cells, is proposed. The introduced deep ESN architecture is trained to allow each UAV to map each observation of the network state to an action, with the goal of minimizing a sequence of time-dependent utility functions. Each UAV uses the ESN to learn its optimal path, transmission power, and cell association vector at different locations along its path. The proposed algorithm is shown to reach a subgame perfect Nash equilibrium upon convergence. Moreover, an upper bound and a lower bound for the altitude of the UAVs are derived thus reducing the computational complexity of the proposed algorithm. The simulation results show that the proposed scheme achieves better wireless latency per UAV and rate per ground user (UE) while requiring a number of steps that are comparable to a heuristic baseline that considers moving via the shortest distance toward the corresponding destinations. The results also show that the optimal altitude of the UAVs varies based on the ground network density and the UE data rate requirements and plays a vital role in minimizing the interference level on the ground UEs as well as the wireless transmission delay of the UAV.read more
Citations
More filters
Journal ArticleDOI
Accessing From the Sky: A Tutorial on UAV Communications for 5G and Beyond
Yong Zeng,Qingqing Wu,Rui Zhang +2 more
TL;DR: In this article, the authors give a tutorial overview of the recent advances in UAV communications to address the above issues, with an emphasis on how to integrate UAVs into the forthcoming fifth-generation (5G) and future cellular networks.
Journal ArticleDOI
A Prospective Look: Key Enabling Technologies, Applications and Open Research Topics in 6G Networks
Lina Bariah,Lina Mohjazi,Sami Muhaidat,Paschalis C. Sofotasios,Gunes Karabulut Kurt,Halim Yanikomeroglu,Octavia A. Dobre +6 more
TL;DR: In this paper, the authors shed light on some of the major enabling technologies for 6G, which are expected to revolutionize the fundamental architectures of cellular networks and provide multiple homogeneous artificial intelligence-empowered services, including distributed communications, control, computing, sensing and energy, from its core to its end nodes.
Journal ArticleDOI
A Survey on Machine-Learning Techniques for UAV-Based Communications
Petros S. Bithas,Emmanouel T. Michailidis,Nikolaos Nomikos,Demosthenes Vouyioukas,Athanasios G. Kanatas +4 more
TL;DR: This article provides a detailed survey of all relevant research works, in which ML techniques have been used on UAV-based communications for improving various design and functional aspects such as channel modeling, resource management, positioning, and security.
Journal ArticleDOI
5G support for Industrial IoT Applications - Challenges, Solutions, and Research gaps
Pal Varga,Jozsef Peto,Attila Franko,David Balla,David Haja,Ferenc Nandor Janky,Gabor Soos,Daniel Ficzere,Markosz Maliosz,Laszlo Toka +9 more
TL;DR: This paper identifies current research challenges and solutions in relation to 5G-enabled Industrial IoT, based on the initial requirements and promises of both domains, and provides meaningful comparisons for each of these areas to draw conclusions on current research gaps.
Journal ArticleDOI
Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
TL;DR: In this article, a comprehensive survey of the state-of-the-art research on DRL for autonomous IoT is presented, where the existing works are classified and summarized under the umbrella of the proposed general DRL model.
References
More filters
Book
Introduction to Reinforcement Learning
TL;DR: In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning.
Book
Data networks
TL;DR: Undergraduate and graduate classes in computer networks and wireless communications; undergraduate classes in discrete mathematics, data structures, operating systems and programming languages.
Book
An Introduction to Game Theory
TL;DR: An Introduction to Game Theory International Edition, by Martin J. Osborne, presents the main principles of game theory and shows how they can be used to understand economics, social, political, and biological phenomena as discussed by the authors.
Book
Synchronization Techniques for Digital Receivers
Umberto Mengali,Aldo N. D’Andrea +1 more
TL;DR: The Principles, Methods and Performance Limits of Carrier Frequency Recovery with Linear Modulations and Timing Recovery with CPM Modulations are presented.
Proceedings ArticleDOI
Modeling air-to-ground path loss for low altitude platforms in urban environments
TL;DR: A statistical propagation model is proposed for predicting the air-to-ground path loss between a low altitude platform and a terrestrial terminal based on the urban environment properties, and is dependent on the elevation angle between the terminal and the platform.