Message Passing Based Distributed Learning for Joint Resource Allocation in Millimeter Wave Heterogeneous Networks

TLDR: This paper studies the joint power control and user association problem in heterogeneous networks (HetNets) by considering the dynamics of links as a Markov Decision Process (MDP), and a reinforcement learning framework is proposed to study the problem.

Abstract: Millimeter wave (mmWave) provides an enormous spectrum for future broadband cellular communications. The corresponding challenging characteristics of radio propagation, and the use of highly directional transmission and the dense deployment, lead to more complex and critical resource allocation problems than in traditional cellular systems, where the channels are better tamed. In this paper, we study the joint power control and user association problem in heterogeneous networks (HetNets) by considering the dynamics of links as a Markov Decision Process (MDP). A reinforcement learning framework is proposed to study the problem. The large state/action space is handled by decomposing the large scale problem into multiple local problems, based on the topology of mmWave HetNet, which is motivated by the celebrated belief propagation (BP) algorithm in probabilistic graphical models. The decomposed problem is solved with a distributed message passing method and accelerated by the prior knowledge of the mmWave dynamics. Two categories of learning frameworks are proposed for time sensitive and power sensitive conditions. The real-world measurements in the 60GHz band are collected and used in the simulation of the proposed framework.