This paper investigates the spectrum sharing problem in vehicular networks based on multi-agent reinforcement learning, where multiple vehicle-to-vehicle (V2V) links reuse the frequency spectrum preoccupied by vehicle-to-infrastructure (V2I) links. Fast channel variations in high mobility vehicular environments preclude the possibility of collecting accurate instantaneous channel state information at the base station for centralized resource management. In response, we model the resource sharing as a multi-agent reinforcement learning problem, which is then solved using a fingerprint-based deep Q-network method that is amenable to a distributed implementation. The V2V links, each acting as an agent, collectively interact with the communication environment, receive distinctive observations yet a common reward, and learn to improve spectrum and power allocation through updating Q-networks using the gained experiences. We demonstrate that with a proper reward design and training mechanism, the multiple V2V agents successfully learn to cooperate in a distributed way to simultaneously improve the sum capacity of V2I links and payload delivery rate of V2V links.

Spectrum Sharing in Vehicular Networks Based on Multi-Agent Reinforcement Learning

As wireless networks evolve toward high mobility and providing better support for connected vehicles, a number of new challenges arise due to the resulting high dynamics in vehicular environments and thus motive rethinking of traditional wireless design methodologies. Future intelligent vehicles, which are at the heart of high mobility networks, are increasingly equipped with multiple advanced onboard sensors and keep generating large volumes of data. Machine learning, as an effective approach to artificial intelligence, can provide a rich set of tools to exploit such data for the benefit of the networks. In this paper, we first identify the distinctive characteristics of high mobility vehicular networks and motivate the use of machine learning to address the resulting challenges. After a brief introduction of the major concepts of machine learning, we discuss its applications to learn the dynamics of vehicular networks and make informed decisions to optimize network performance. In particular, we discuss in greater detail the application of reinforcement learning in managing network resources as an alternative to the prevalent optimization approach. Finally, some open issues worth further investigation are highlighted.

Toward Intelligent Vehicular Networks: A Machine Learning Framework

Internet-of-Things (IoT) refers to a massively heterogeneous network formed through smart devices connected to the Internet. In the wake of disruptive IoT with a huge amount and variety of data, Machine Learning (ML) and Deep Learning (DL) mechanisms will play a pivotal role to bring intelligence to the IoT networks. Among other aspects, ML and DL can play an essential role in addressing the challenges of resource management in large-scale IoT networks. In this article, we conduct a systematic and in-depth survey of the ML- and DL-based resource management mechanisms in cellular wireless and IoT networks. We start with the challenges of resource management in cellular IoT and low-power IoT networks, review the traditional resource management mechanisms for IoT networks, and motivate the use of ML and DL techniques for resource management in these networks. Then, we provide a comprehensive survey of the existing ML- and DL-based resource management techniques in wireless IoT networks and the techniques specifically designed for HetNets, MIMO and D2D communications, and NOMA networks. To this end, we also identify the future research directions in using ML and DL for resource allocation and management in IoT networks.

Machine Learning for Resource Management in Cellular and IoT Networks: Potentials, Current Solutions, and Open Challenges

A wide variety of works have been conducted in vehicle-to-everything (V2X) communications to enable a variety of applications for road safety, traffic efficiency and passenger infotainment. Although dedicated short-range communications (DSRC) based V2X is already in the deployment phase, cellular based V2X is gaining more interest in academia and industry most recently. This article surveys the existing work and challenges on LTE and 5G to support efficient V2X communications. First, we present the motivations for cellular based V2X communications. Second, we summarize the LTE V2X architecture and operating scenarios being considered. Third, we discuss the challenges in existing LTE for supporting V2X communications such as physical layer structure, synchronization, multimedia broadcast multicast services (MBMS), resource allocation, security and survey the recent solutions to these challenges. We further discuss the challenges and possible solutions for 5G based vehicular communications. Finally, we discuss the open research issues and possible research directions in cellular based vehicular communications.

Challenges and Solutions for Cellular Based V2X Communications

It has been a long-held belief that judicious resource allocation is critical to mitigating interference, improving network efficiency, and ultimately optimizing wireless communication performance. The traditional wisdom is to explicitly formulate resource allocation as an optimization problem and then exploit mathematical programming to solve the problem to a certain level of optimality. Nonetheless, as wireless networks become increasingly diverse and complex, for example, in the high-mobility vehicular networks, the current design methodologies face significant challenges and thus call for rethinking of the traditional design philosophy. Meanwhile, deep learning, with many success stories in various disciplines, represents a promising alternative due to its remarkable power to leverage data for problem solving. In this article, we discuss the key motivations and roadblocks of using deep learning for wireless resource allocation with application to vehicular networks. We review major recent studies that mobilize the deep-learning philosophy in wireless resource allocation and achieve impressive results. We first discuss deep-learning-assisted optimization for resource allocation. We then highlight the deep reinforcement learning approach to address resource allocation problems that are difficult to handle in the traditional optimization framework. We also identify some research directions that deserve further investigation.

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

This paper investigates the resource allocation problem in device-to-device-based vehicular communications, based on slow fading statistics of channel state information (CSI), to alleviate signaling overhead for reporting rapidly varying accurate CSI of mobile links. We consider the case when each vehicle-to-infrastructure (V2I) link shares spectrum with multiple vehicle-to-vehicle (V2V) links. Leveraging the slow fading statistical CSI of mobile links, we maximize the sum V2I capacity while guaranteeing the reliability of all V2V links. We use graph partitioning tools to divide highly interfering V2V links into different clusters before formulating the spectrum sharing problem as a weighted 3-D matching problem. We propose a suite of algorithms, including a baseline graph-based resource allocation algorithm, a greedy resource allocation algorithm, and a randomized resource allocation algorithm, to address the performance-complexity tradeoffs. We further investigate resource allocation adaption in response to slow fading CSI of all vehicular links and develop a low-complexity randomized algorithm.

Graph-Based Resource Sharing in Vehicular Communication

This paper investigates the resource allocation problem in device-to-device (D2D)-based vehicular communications, based on slow fading statistics of channel state information (CSI), to alleviate signaling overhead for reporting rapidly varying accurate CSI of mobile links. We consider the case when each vehicle-to-infrastructure (V2I) link shares spectrum with multiple vehicle-to-vehicle (V2V) links. Leveraging the slow fading statistical CSI of mobile links, we maximize the sum V2I capacity while guaranteeing the reliability of all V2V links. We propose a graph- based algorithm that uses graph partitioning tools to divide highly interfering V2V links into different clusters before formulating the spectrum sharing problem as a weighted 3-dimensional matching problem, which is then solved through adapting a high-performance approximation algorithm.

Graph-Based Radio Resource Management for Vehicular Networks

This paper investigates the resource allocation problem in device-to-device (D2D)-based vehicular communications, based on slow fading statistics of channel state information (CSI), to alleviate signaling overhead for reporting rapidly varying accurate CSI of mobile links. We consider the case when each vehicle-to-infrastructure (V2I) link shares spectrum with multiple vehicle-to-vehicle (V2V) links. Leveraging the slow fading statistical CSI of mobile links, we maximize the sum V2I capacity while guaranteeing the reliability of all V2V links. We propose a graph-based algorithm that uses graph partitioning tools to divide highly interfering V2V links into different clusters before formulating the spectrum sharing problem as a weighted 3-dimensional matching problem, which is then solved through adapting a high-performance approximation algorithm.

We say that a graph $G$ is $(2,m)$-linked if, for any distinct vertices $a_1,\ldots, a_m, b_1,b_2$ in $G$, there exist vertex disjoint connected subgraphs $A,B$ of $G$ such that $\{a_1, \ldots, a_m\}$ is contained in $A$ and $\{b_1,b_2\}$ is contained in $B$. A fundamental result in structural graph theory is the characterization of $(2,2)$-linked graphs, with different versions obtained independently by Robertson and Chakravarty, Seymour, and Thomassen. It appears to be very difficult to characterize $(2,m)$-linked graphs for $m\ge 3$. In this paper, we provide a partial characterization of $(2,m)$-linked graphs by adding an average degree condition. This implies that $(2m+2)$-connected graphs are $(2,m)$-linked. Moreover, if $G$ is a $(2m+2)$-connected graph and $a_1, \ldots, a_m, b_1,b_2$ are distinct vertices of $G$, then there is a path $P$ in $G$ between $b_1$ and $b_2$ and avoiding $\{a_1, \ldots, a_m\}$ such that $G-P$ is connected, improving a previous connectivity bound of $10m$.

Shijie Xie

Papers

Graph-Based Resource Sharing in Vehicular Communication

Graph-Based Radio Resource Management for Vehicular Networks

Graph-Based Radio Resource Management for Vehicular Networks

Linkages and removable paths avoiding vertices