This paper presents a comprehensive literature review on applications of deep reinforcement learning (DRL) in communications and networking. Modern networks, e.g., Internet of Things (IoT) and unmanned aerial vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, DRL, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of DRL from fundamental concepts to advanced models. Then, we review DRL approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks, such as 5G and beyond. Furthermore, we present applications of DRL for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying DRL.

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

The rapid uptake of mobile devices and the rising popularity of mobile applications and services pose unprecedented demands on mobile and wireless networking infrastructure. Upcoming 5G systems are evolving to support exploding mobile traffic volumes, real-time extraction of fine-grained analytics, and agile management of network resources, so as to maximize user experience. Fulfilling these tasks is challenging, as mobile environments are increasingly complex, heterogeneous, and evolving. One potential solution is to resort to advanced machine learning techniques, in order to help manage the rise in data volumes and algorithm-driven applications. The recent success of deep learning underpins new and powerful tools that tackle problems in this space. In this paper, we bridge the gap between deep learning and mobile and wireless networking research, by presenting a comprehensive survey of the crossovers between the two areas. We first briefly introduce essential background and state-of-the-art in deep learning techniques with potential applications to networking. We then discuss several techniques and platforms that facilitate the efficient deployment of deep learning onto mobile systems. Subsequently, we provide an encyclopedic review of mobile and wireless networking research based on deep learning, which we categorize by different domains. Drawing from our experience, we discuss how to tailor deep learning to mobile environments. We complete this survey by pinpointing current challenges and open future directions for research.

/pdf/deep-learning-in-mobile-and-wireless-networking-a-survey-ct8cy2kuqi.pdf

Deep Learning in Mobile and Wireless Networking: A Survey

The use of unmanned aerial vehicles (UAVs) is growing rapidly across many civil application domains, including real-time monitoring, providing wireless coverage, remote sensing, search and rescue, delivery of goods, security and surveillance, precision agriculture, and civil infrastructure inspection. Smart UAVs are the next big revolution in the UAV technology promising to provide new opportunities in different applications, especially in civil infrastructure in terms of reduced risks and lower cost. Civil infrastructure is expected to dominate more than $45 Billion market value of UAV usage. In this paper, we present UAV civil applications and their challenges. We also discuss the current research trends and provide future insights for potential UAV uses. Furthermore, we present the key challenges for UAV civil applications, including charging challenges, collision avoidance and swarming challenges, and networking and security-related challenges. Based on our review of the recent literature, we discuss open research challenges and draw high-level insights on how these challenges might be approached.

Unmanned Aerial Vehicles: A Survey on Civil Applications and Key Research Challenges

Distributed denial of service (DDoS) attacks in cloud computing environments are growing due to the essential characteristics of cloud computing. With recent advances in software-defined networking (SDN), SDN-based cloud brings us new chances to defeat DDoS attacks in cloud computing environments. Nevertheless, there is a contradictory relationship between SDN and DDoS attacks. On one hand, the capabilities of SDN, including software-based traffic analysis, centralized control, global view of the network, dynamic updating of forwarding rules, make it easier to detect and react to DDoS attacks. On the other hand, the security of SDN itself remains to be addressed, and potential DDoS vulnerabilities exist across SDN platforms. In this paper, we discuss the new trends and characteristics of DDoS attacks in cloud computing, and provide a comprehensive survey of defense mechanisms against DDoS attacks using SDN. In addition, we review the studies about launching DDoS attacks on SDN, as well as the methods against DDoS attacks in SDN. To the best of our knowledge, the contradictory relationship between SDN and DDoS attacks has not been well addressed in previous works. This work can help to understand how to make full use of SDN's advantages to defeat DDoS attacks in cloud computing environments and how to prevent SDN itself from becoming a victim of DDoS attacks, which are important for the smooth evolution of SDN-based cloud without the distraction of DDoS attacks.

Software-Defined Networking (SDN) and Distributed Denial of Service (DDoS) Attacks in Cloud Computing Environments: A Survey, Some Research Issues, and Challenges

The fifth generation (5G) wireless network technology is to be standardized by 2020, where main goals are to improve capacity, reliability, and energy efficiency, while reducing latency and massively increasing connection density. An integral part of 5G is the capability to transmit touch perception type real-time communication empowered by applicable robotics and haptics equipment at the network edge. In this regard, we need drastic changes in network architecture including core and radio access network (RAN) for achieving end-to-end latency on the order of 1 ms. In this paper, we present a detailed survey on the emerging technologies to achieve low latency communications considering three different solution domains: 1) RAN; 2) core network; and 3) caching. We also present a general overview of major 5G cellular network elements such as software defined network, network function virtualization, caching, and mobile edge computing capable of meeting latency and other 5G requirements.

A Survey on Low Latency Towards 5G: RAN, Core Network and Caching Solutions

Since wireless network virtualization enables abstraction and sharing of infrastructure and radio spectrum resources, the overall expenses of wireless network deployment and operation can be reduced significantly. Moreover, wireless network virtualization can provide easier migration to newer products or technologies by isolating part of the network. Despite the potential vision of wireless network virtualization, several significant research challenges remain to be addressed before widespread deployment of wireless network virtualization, including isolation, control signaling, resource discovery and allocation, mobility management, network management and operation, and security as well as non-technical issues such as governance regulations, etc. In this paper, we provide a brief survey on some of the works that have already been done to achieve wireless network virtualization, and discuss some research issues and challenges. We identify several important aspects of wireless network virtualization: overview, motivations, framework, performance metrics, enabling technologies, and challenges. Finally, we explore some broader perspectives in realizing wireless network virtualization.

Wireless Network Virtualization: A Survey, Some Research Issues and Challenges

Mobile edge computing has risen as a promising technology for augmenting the computational capabilities of mobile devices Meanwhile, in-network caching has become a natural trend of the solution of handling exponentially increasing Internet traffic The important issues in these two networking paradigms are computation offloading and content caching strategies, respectively In order to jointly tackle these issues in wireless cellular networks with mobile edge computing, we formulate the computation offloading decision, resource allocation and content caching strategy as an optimization problem, considering the total revenue of the network Furthermore, we transform the original problem into a convex problem and then decompose it in order to solve it in a distributed and efficient way Finally, with recent advances in distributed convex optimization, we develop an alternating direction method of multipliers-based algorithm to solve the optimization problem The effectiveness of the proposed scheme is demonstrated by simulation results with different system parameters

Computation Offloading and Resource Allocation in Wireless Cellular Networks With Mobile Edge Computing

Mobile edge computing (MEC) has attracted great interests as a promising approach to augment computational capabilities of mobile devices. An important issue in the MEC paradigm is computation offloading. In this paper, we propose an integrated framework for computation offloading and interference management in wireless cellular networks with MEC. In this integrated framework, we formulate the computation offloading decision, physical resource block (PRB) allocation, and MEC computation resource allocation as optimization problems. The MEC server makes the offloading decision according to the local computation overhead estimated by all user equipments (UEs) and the offloading overhead estimated by the MEC server itself. Then, the MEC server performs the PRB allocation using the graph coloring method. The outcomes of the offloading decision and PRB allocation are then used to distribute the computation resource of the MEC server to the UEs. Simulation results are presented to show the effectiveness of the proposed scheme with different system parameters.

Joint Computation Offloading and Interference Management in Wireless Cellular Networks with Mobile Edge Computing

Wireless network virtualization and information-centric networking (ICN) are two promising techniques in software-defined 5G mobile wireless networks. Traditionally, these two technologies have been addressed separately. In this paper we show that integrating wireless network virtualization with ICN techniques can significantly improve the end-to-end network performance. In particular, we propose an information- centric wireless network virtualization architecture for integrating wireless network virtualization with ICN. We develop the key components of this architecture: radio spectrum resource, wireless network infrastructure, virtual resources (including content-level slicing, network-level slicing, and flow-level slicing), and informationcentric wireless virtualization controller. Then we formulate the virtual resource allocation and in-network caching strategy as an optimization problem, considering the gain of not only virtualization but also in-network caching in our proposed information-centric wireless network virtualization architecture. The obtained simulation results show that our proposed information-centric wireless network virtualization architecture and the related schemes significantly outperform the other existing schemes.

Information-centric network function virtualization over 5g mobile wireless networks

With wireless virtualization, the overall expenses of mobile cellular network deployment and operation can be significantly reduced by abstracting and sharing infrastructure and radio spectrum resources. Moreover, wireless virtualization can provide easier migration to newer products or technologies by isolating part of the network. Despite the potential vision of wireless virtualization, several significant research challenges remain. In this article, we provide a brief survey on some of the work that has been done on wireless virtualization in cellular networks. We also present the motivations and business models of wireless virtualization. Furthermore, we present a framework that enables wireless virtualization. In addition, we discuss a number of challenges that need to be addressed for the deployment of wireless virtualization in next generation mobile cellular networks.

Chengchao Liang

Papers

Wireless Network Virtualization: A Survey, Some Research Issues and Challenges

Computation Offloading and Resource Allocation in Wireless Cellular Networks With Mobile Edge Computing

Joint Computation Offloading and Interference Management in Wireless Cellular Networks with Mobile Edge Computing

Information-centric network function virtualization over 5g mobile wireless networks

Wireless virtualization for next generation mobile cellular networks