Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access
Oshri Naparstek,Kobi Cohen +1 more
Reads0
Chats0
TLDR
A novel distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning is developed for accessing the spectrum that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users.Abstract:
We consider the problem of dynamic spectrum access for network utility maximization in multichannel wireless networks. The shared bandwidth is divided into $K$ orthogonal channels. In the beginning of each time slot, each user selects a channel and transmits a packet with a certain transmission probability. After each time slot, each user that has transmitted a packet receives a local observation indicating whether its packet was successfully delivered or not (i.e., ACK signal). The objective is a multi-user strategy for accessing the spectrum that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users. Obtaining an optimal solution for the spectrum access problem is computationally expensive, in general, due to the large-state space and partial observability of the states. To tackle this problem, we develop a novel distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning. Specifically, at each time slot, each user maps its current state to the spectrum access actions based on a trained deep-Q network used to maximize the objective function. Game theoretic analysis of the system dynamics is developed for establishing design principles for the implementation of the algorithm. The experimental results demonstrate the strong performance of the algorithm.read more
Citations
More filters
Journal ArticleDOI
Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey
TL;DR: A comprehensive survey of the applications of DL algorithms for different network layers, including physical layer modulation/coding, data link layer access control/resource allocation, and routing layer path search, and traffic balancing is performed.
Journal ArticleDOI
Deep Reinforcement Learning for User Association and Resource Allocation in Heterogeneous Cellular Networks
TL;DR: A reinforcement learning approach is proposed to achieve the maximum long-term overall network utility while guaranteeing the quality of service requirements of user equipments (UEs) in the downlink of heterogeneous cellular networks.
Journal ArticleDOI
Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks
Yasar Sinan Nasir,Dongning Guo +1 more
TL;DR: The proposed algorithm is shown to achieve near-optimal power allocation in real time based on delayed CSI measurements available to the agents and is especially suitable for practical scenarios where the system model is inaccurate and CSI delay is non-negligible.
Journal ArticleDOI
Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks
Yasar Sinan Nasir,Dongning Guo +1 more
TL;DR: In this paper, a distributively executed dynamic power allocation scheme is developed based on model-free deep RL for transmit power control in wireless networks, where each transmitter collects CSI and quality of service (QoS) information from several neighbors and adapts its own transmit power accordingly.
Journal ArticleDOI
Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks
TL;DR: The key motivations and roadblocks of using deep learning for wireless resource allocation with application to vehicular networks are discussed and the deep reinforcement learning approach to address resource allocation problems that are difficult to handle in the traditional optimization framework is highlighted.
References
More filters
Journal ArticleDOI
Human-level control through deep reinforcement learning
Volodymyr Mnih,Koray Kavukcuoglu,David Silver,Andrei Rusu,Joel Veness,Marc G. Bellemare,Alex Graves,Martin Riedmiller,Andreas K. Fidjeland,Georg Ostrovski,Stig Petersen,Charles Beattie,Amir Sadik,Ioannis Antonoglou,Helen King,Dharshan Kumaran,Daan Wierstra,Shane Legg,Demis Hassabis +18 more
TL;DR: This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.
Posted Content
Playing Atari with Deep Reinforcement Learning
Volodymyr Mnih,Koray Kavukcuoglu,David Silver,Alex Graves,Ioannis Antonoglou,Daan Wierstra,Martin Riedmiller +6 more
TL;DR: This work presents the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning, which outperforms all previous approaches on six of the games and surpasses a human expert on three of them.
Journal ArticleDOI
A Survey of Dynamic Spectrum Access
Qing Zhao,Brian M. Sadler +1 more
TL;DR: An overview of challenges and recent developments in both technological and regulatory aspects of opportunistic spectrum access (OSA) is presented, and the three basic components of OSA are discussed.
Proceedings Article
Deep reinforcement learning with double Q-Learning
TL;DR: In this paper, the authors show that the DQN algorithm suffers from substantial overestimation in some games in the Atari 2600 domain, and they propose a specific adaptation to the algorithm and show that this algorithm not only reduces the observed overestimations, but also leads to much better performance on several games.
Journal ArticleDOI
Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: A POMDP framework
TL;DR: An analytical framework for opportunistic spectrum access based on the theory of partially observable Markov decision process (POMDP) is developed and cognitive MAC protocols that optimize the performance of secondary users while limiting the interference perceived by primary users are proposed.