This paper presents a comprehensive literature review on applications of deep reinforcement learning (DRL) in communications and networking. Modern networks, e.g., Internet of Things (IoT) and unmanned aerial vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, DRL, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of DRL from fundamental concepts to advanced models. Then, we review DRL approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks, such as 5G and beyond. Furthermore, we present applications of DRL for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying DRL.

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

As a promising machine learning tool to handle the accurate pattern recognition from complex raw data, deep learning (DL) is becoming a powerful method to add intelligence to wireless networks with large-scale topology and complex radio conditions. DL uses many neural network layers to achieve a brain-like acute feature extraction from high-dimensional raw data. It can be used to find the network dynamics (such as hotspots, interference distribution, congestion points, traffic bottlenecks, spectrum availability, etc.) based on the analysis of a large amount of network parameters (such as delay, loss rate, link signal-to-noise ratio, etc.). Therefore, DL can analyze extremely complex wireless networks with many nodes and dynamic link quality. This paper performs a comprehensive survey of the applications of DL algorithms for different network layers, including physical layer modulation/coding, data link layer access control/resource allocation, and routing layer path search, and traffic balancing. The use of DL to enhance other network functions, such as network security, sensing data compression, etc., is also discussed. Moreover, the challenging unsolved research issues in this field are discussed in detail, which represent the future research trends of DL-based wireless networks. This paper can help the readers to deeply understand the state-of-the-art of the DL-based wireless network designs, and select interesting unsolved issues to pursue in their research.

Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey

The ubiquitous deployment of mobile and sensor devices is creating a new environment, namely the Internet of Things(IoT), that enables a wide range of future Internet applications. In this work, we present Mobile Fog, a high level programming model for the future Internet applications that are geospatially distributed, large-scale, and latency-sensitive. We analyze use cases for the programming model with camera network and connected vehicle applications to show the efficacy of Mobile Fog. We also evaluate application performance through simulation.

Mobile fog: a programming model for large-scale applications on the internet of things

A widespread methodology for performance analysis in the field of communication systems engineering is network simulation. While ns-2 has established itself as virtually the standard network simulation tool, other network simulators have gained more and more attention during the last years. In this paper, we briefly survey new developments in the field of network simulation and conduct a performance comparison study by implementing an identical simulation set-up in five simulators, namely ns-2, OMNet++, ns-3, SimPy and JiST/SWANS. Our results reveal large differences according to both run-time performance and memory usage.

A Performance Comparison of Recent Network Simulators

Vehicular ad hoc networks (VANETs) are becoming the most promising research topic in intelligent transportation systems, because they provide information to deliver comfort and safety to both drivers and passengers. However, unique characteristics of VANETs make security, privacy, and trust management challenging issues in VANETs’ design. This survey article starts with the necessary background of VANETs, followed by a brief treatment of main security services, which have been well studied in other fields. We then focus on an in-depth review of anonymous authentication schemes implemented by five pseudonymity mechanisms. Because of the predictable dynamics of vehicles, anonymity is necessary but not sufficient to thwart tracking an attack that aims at the drivers’ location profiles. Thus, several location privacy protection mechanisms based on pseudonymity are elaborated to further protect the vehicles’ privacy and guarantee the quality of location-based services simultaneously. We also give a comprehensive analysis on various trust management models in VANETs. Finally, considering that current and near-future applications in VANETs are evaluated by simulation, we give a much-needed update on the latest mobility and network simulators as well as the integrated simulation platforms. In sum, this paper is carefully positioned to avoid overlap with existing surveys by filling the gaps and reporting the latest advances in VANETs while keeping it self-explained.

A Survey on Recent Advances in Vehicular Network Security, Trust, and Privacy

The OMNeT++ discrete event simulation environment has been publicly available since 1997. It has been created with the simulation of communication networks, multiprocessors and other distributed systems in mind as application area, but instead of building a specialized simulator, OMNeT++ was designed to be as general as possible. Since then, the idea has proven to work, and OMNeT++ has been used in numerous domains from queuing network simulations to wireless and ad-hoc network simulations, from business process simulation to peer-to-peer network, optical switch and storage area network simulations. This paper presents an overview of the OMNeT++ framework, recent challenges brought about by the growing amount and complexity of third party simulation models, and the solutions we introduce in the next major revision of the simulation framework.

/pdf/an-overview-of-the-omnet-simulation-environment-23lg31mm1p.pdf

Rudolf Hornig

Papers

An overview of the OMNeT++ simulation environment

AN OVERVIEW OF THE OMNeT++ SIMULATION ENVIRONMENT