This paper presents a comprehensive literature review on applications of deep reinforcement learning (DRL) in communications and networking. Modern networks, e.g., Internet of Things (IoT) and unmanned aerial vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, DRL, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of DRL from fundamental concepts to advanced models. Then, we review DRL approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks, such as 5G and beyond. Furthermore, we present applications of DRL for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying DRL.

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

This paper presents a comprehensive literature review on applications of deep reinforcement learning in communications and networking. Modern networks, e.g., Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, deep reinforcement learning, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of deep reinforcement learning from fundamental concepts to advanced models. Then, we review deep reinforcement learning approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks such as 5G and beyond. Furthermore, we present applications of deep reinforcement learning for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying deep reinforcement learning.

Mobile robots are increasingly used in automated industrial environments. There are also other applications like planet exploration, surveillance, landmine detection, etc. In all these applications, in order that the mobile robots perform their tasks, collision-free path planning is a prerequisite. This article provides an overview of the research progress in path planning of a mobile robot for off-line as well as on-line environments. Commonly used classic and evolutionary approaches of path planning of mobile robots have been addressed. Review shows that evolutionary optimization algorithms are computationally efficient and hence are increasingly being used in tandem with classic approaches while handling Non-deterministic Polynomial time hard (NP-hard) problems. Also, challenges involved in developing a computationally efficient path planning algorithm are addressed.

	 

	Key words: Path planning, mobile robot, off-line environment, on-line environment, classic, evolutionary algorithms.

/pdf/optimal-path-planning-of-mobile-robots-a-review-4uumad7sbe.pdf

Optimal path planning of mobile robots: A review

The BPF proposal ensures a feasible, optimal and safe path for robot navigation.The results of BPF overcomes APF and other EAPF methods like those based in GAs.The BPF is quite faster in optimization leading to reduction in computation burden.The BPF running in parallel mode is the most suitable to fulfill local and global controllability.The BPF is capable to work in offline and online mode with static and dynamic obstacles. In this paper, optimal paths in environments with static and dynamic obstacles for a mobile robot (MR) are computed using a new method for path planning. The proposed method called Bacterial Potential Field (BPF) ensures a feasible, optimal and safe path. This novel proposal makes use of the Artificial Potential Field (APF) method with a Bacterial Evolutionary Algorithm (BEA) to obtain an enhanced flexible path planner method taking all the advantages of using the APF method, strongly reducing its disadvantages. Comparative experiments for sequential and parallel implementations of the BPF method against the classic APF method, as well as with the Pseudo-Bacterial Potential Field (PBPF) method, and with the Genetic Potential Field (GPF) method, all of them based on evolutionary computation to optimize the APF parameters, were achieved. A simulation platform that uses an MR realistic model was designed to test the path planning algorithms. In general terms, it was demonstrated that the BPF outperforms the APF, GPF, and the PBPF methods by reducing the computational time to find the optimal path at least by a factor of 1.59. These results have a positive impact in the ability of the BPF path planning method to satisfy local and global controllability in dynamic complex environments, avoiding collisions with objects that will interfere the navigation of the MR.

Path planning for mobile robots using Bacterial Potential Field for avoiding static and dynamic obstacles

Good path planning technology of mobile robot can not only save a lot of time, but also reduce the wear and capital investment of mobile robot. Several methodologies have been proposed and reported in the literature for the path planning of mobile robot. Although these methodologies do not guarantee an optimal solution, they have been successfully applied in their works. The purpose of this paper is to review the modeling, optimization criteria and solution algorithms for the path planning of mobile robot. The survey shows GA (genetic algorithm), PSO (particle swarm optimization algorithm), APF (artificial potential field), and ACO (ant colony optimization algorithm) are the most used approaches to solve the path planning of mobile robot. Finally, future research is discussed which could provide reference for the path planning of mobile robot.

/pdf/path-planning-for-the-mobile-robot-a-review-52e8lq6p4b.pdf

Path Planning for the Mobile Robot: A Review

A new methodology named Evolutionary Artificial Potential Field (EAPF) is proposed for real-time robot path planning. The artificial potential field method is combined with genetic algorithms, to derive optimal potential field functions. The proposed EAPF approach is capable of navigating robot(s) situated among moving obstacles. Potential field functions for obstacles and goal points are also defined. The potential field functions for obstacles contain tunable parameters. The multi-objective evolutionary algorithm (MOEA) is utilized to identify the optimal potential field functions. Fitness functions such as goal-factor, obstacle-factor, smoothness-factor and minimum-pathlength-factor are developed for the MOEA selection criteria. An algorithm named escape-force is introduced to avoid the local minima associated with EAPF. Moving obstacles and moving goal positions were considered to test the robust performance of the proposed methodology. Simulation results show that the proposed methodology is efficient and robust for robot path planning with non-stationary goals and obstacles.

Wang Ming-Liang

Papers

Evolutionary artificial potential fields and their application in real time robot path planning