Wireless powered mobile-edge computing (MEC) has recently emerged as a promising paradigm to enhance the data processing capability of low-power networks, such as wireless sensor networks and internet of things (IoT). In this paper, we consider a wireless powered MEC network that adopts a binary offloading policy, so that each computation task of wireless devices (WDs) is either executed locally or fully offloaded to an MEC server. Our goal is to acquire an online algorithm that optimally adapts task offloading decisions and wireless resource allocations to the time-varying wireless channel conditions. This requires quickly solving hard combinatorial optimization problems within the channel coherence time, which is hardly achievable with conventional numerical optimization methods. To tackle this problem, we propose a Deep Reinforcement learning-based Online Offloading (DROO) framework that implements a deep neural network as a scalable solution that learns the binary offloading decisions from the experience. It eliminates the need of solving combinatorial optimization problems, and thus greatly reduces the computational complexity especially in large-size networks. To further reduce the complexity, we propose an adaptive procedure that automatically adjusts the parameters of the DROO algorithm on the fly. Numerical results show that the proposed algorithm can achieve near-optimal performance while significantly decreasing the computation time by more than an order of magnitude compared with existing optimization methods. For example, the CPU execution latency of DROO is less than 0.1 second in a 30-user network, making real-time and optimal offloading truly viable even in a fast fading environment.

Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks

Wireless powered mobile-edge computing (MEC) has recently emerged as a promising paradigm to enhance the data processing capability of low-power networks, such as wireless sensor networks and internet of things (IoT). In this paper, we consider a wireless powered MEC network that adopts a binary offloading policy, so that each computation task of wireless devices (WDs) is either executed locally or fully offloaded to an MEC server. Our goal is to acquire an online algorithm that optimally adapts task offloading decisions and wireless resource allocations to the time-varying wireless channel conditions. This requires quickly solving hard combinatorial optimization problems within the channel coherence time, which is hardly achievable with conventional numerical optimization methods. To tackle this problem, we propose a Deep Reinforcement learning-based Online Offloading (DROO) framework that implements a deep neural network as a scalable solution that learns the binary offloading decisions from the experience. It eliminates the need of solving combinatorial optimization problems, and thus greatly reduces the computational complexity especially in large-size networks. To further reduce the complexity, we propose an adaptive procedure that automatically adjusts the parameters of the DROO algorithm on the fly. Numerical results show that the proposed algorithm can achieve near-optimal performance while significantly decreasing the computation time by more than an order of magnitude compared with existing optimization methods. For example, the CPU execution latency of DROO is less than $0.1$ second in a $30$-user network, making real-time and optimal offloading truly viable even in a fast fading environment.

Driven by the emergence of new compute-intensive applications and the vision of the Internet of Things (IoT), it is foreseen that the emerging 5G network will face an unprecedented increase in traffic volume and computation demands. However, end users mostly have limited storage capacities and finite processing capabilities, thus how to run compute-intensive applications on resource-constrained users has recently become a natural concern. Mobile edge computing (MEC), a key technology in the emerging fifth generation (5G) network, can optimize mobile resources by hosting compute-intensive applications, process large data before sending to the cloud, provide the cloud computing capabilities within the radio access network (RAN) in close proximity to mobile users, and offer context-aware services with the help of RAN information. Therefore, MEC enables a wide variety of applications, where the real-time response is strictly required, e.g., driverless vehicles, augmented reality, robotics, and immerse media. Indeed, the paradigm shift from 4G to 5G could become a reality with the advent of new technological concepts. The successful realization of MEC in the 5G network is still in its infancy and demands for constant efforts from both academic and industry communities. In this survey, we first provide a holistic overview of MEC technology and its potential use cases and applications. Then, we outline up-to-date researches on the integration of MEC with the new technologies that will be deployed in 5G and beyond. We also summarize testbeds and experimental evaluations, and open source activities, for edge computing. We further summarize lessons learned from state-of-the-art research works as well as discuss challenges and potential future directions for MEC research.

A Survey of Multi-Access Edge Computing in 5G and Beyond: Fundamentals, Technology Integration, and State-of-the-Art

Fog/Edge computing emerges as a novel computing paradigm that harnesses resources in the proximity of the Internet of Things (IoT) devices so that, alongside with the cloud servers, provide services in a timely manner. However, due to the ever-increasing growth of IoT devices with resource-hungry applications, fog/edge servers with limited resources cannot efficiently satisfy the requirements of the IoT applications. Therefore, the application placement in the fog/edge computing environment, in which several distributed fog/edge servers and centralized cloud servers are available, is a challenging issue. In this article, we propose a weighted cost model to minimize the execution time and energy consumption of IoT applications, in a computing environment with multiple IoT devices, multiple fog/edge servers and cloud servers. Besides, a new application placement technique based on the Memetic Algorithm is proposed to make batch application placement decision for concurrent IoT applications. Due to the heterogeneity of IoT applications, we also propose a lightweight pre-scheduling algorithm to maximize the number of parallel tasks for the concurrent execution. The performance results demonstrate that our technique significantly improves the weighted cost of IoT applications up to 65 percent in comparison to its counterparts.

An Application Placement Technique for Concurrent IoT Applications in Edge and Fog Computing Environments

In recent years, computation offloading has become an effective way to overcome the constraints of mobile devices (MDs) by offloading delay-sensitive and computation-intensive mobile application tasks to remote cloud-based data centers. Smart cities can benefit from offloading to edge points in the framework of the so-called cyber–physical–social systems (CPSS), as for example in traffic violation tracking cameras. We assume that there are mobile edge computing networks (MECNs) in more than one region, and they consist of multiple access points, multi-edge servers, and $N$ MDs, where each MD has $M$ independent real-time massive tasks. The MDs can connect to a MECN through the access points or the mobile network. Each task be can processed locally by the MD itself or remotely. There are three offloading options: nearest edge server, adjacent edge server, and remote cloud. We propose a reinforcement-learning-based state-action-reward-state-action (RL-SARSA) algorithm to resolve the resource management problem in the edge server, and make the optimal offloading decision for minimizing system cost, including energy consumption and computing time delay. We call this method OD-SARSA (offloading decision-based SARSA). We compared our proposed method with reinforcement learning based Q learning (RL-QL), and it is concluded that the performance of the former is superior to that of the latter.

/pdf/task-offloading-and-resource-allocation-for-mobile-edge-1jbhpmfbax.pdf

Task Offloading and Resource Allocation for Mobile Edge Computing by Deep Reinforcement Learning Based on SARSA

Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing

This paper studies mobile edge computing (MEC) networks where multiple wireless devices (WDs) choose to offload their computation tasks to an edge server. To conserve energy and maintain quality of service for WDs, the optimization of joint offloading decision and bandwidth allocation is formulated as a mixed integer programming problem. However, the problem is computationally limited by the curse of dimensionality, which cannot be solved by general optimization tools in an effective and efficient way, especially for large-scale WDs. In this paper, we propose a distributed deep learning-based offloading (DDLO) algorithm for MEC networks, where multiple parallel DNNs are used to generate offloading decisions. We adopt a shared replay memory to store newly generated offloading decisions which are further to train and improve all DNNs. Extensive numerical results show that the proposed DDLO algorithm can generate near-optimal offloading decisions in less than one second.

Distributed Deep Learning-based Offloading for Mobile Edge Computing Networks

This paper studies mobile edge computing (MEC) networks where multiple wireless devices (WDs) offload their computation tasks to multiple edge servers and one cloud server. Considering different real-time computation tasks at different WDs, every task is decided to be processed locally at its WD or to be offloaded to and processed at one of the edge servers or the cloud server. In this paper, we investigate low-complexity computation offloading policies to guarantee quality of service of the MEC network and to minimize WDs’ energy consumption. Specifically, both a linear programing relaxation-based (LR-based) algorithm and a distributed deep learning-based offloading (DDLO) algorithm are independently studied for MEC networks. We further propose a heterogeneous DDLO to achieve better convergence performance than DDLO. Extensive numerical results show that the DDLO algorithms guarantee better performance than the LR-based algorithm. Furthermore, the DDLO algorithm generates an offloading decision in less than 1 millisecond, which is several orders faster than the LR-based algorithm.

https://www.mdpi.com/1424-8220/19/6/1446/pdf

Multi-Server Multi-User Multi-Task Computation Offloading for Mobile Edge Computing Networks

We consider a mobile edge computing system that every user has multiple tasks being offloaded to edge server via wireless networks. Our goal is to acquire a satisfactory task offloading and resource allocation decision for each user so as to minimize energy consumption and delay. In this paper, we propose a deep reinforcement learning-based approach to solve joint task offloading and resource allocation problems. Simulation results show that the proposed deep Q-learning-based algorithm can achieve near-optimal performance.

Deep Reinforcement Learning-Based Task Offloading and Resource Allocation for Mobile Edge Computing

Powering relays with harvested renewable ambient energy has been emerging as a promising solution to reduce the on-grid energy consumption and greenhouse gas emissions in green relaying communication networks. In this paper, we study the joint time scheduling and power allocation problem for the Decode-and-Forward energy-harvesting relay communication network. Particularly, our goal is to maximize the end-to-end throughput by a deadline subject to the finite data and energy storage. Due to the multi-slot optimization, the traditional deep reinforcement learning (RL) framework cannot be directly applied to obtain the optimal solution of maximizing the end-to-end throughput by a deadline in the online manner. To this end, we explore a novel deep reinforcement learning framework consisting of multiple computation units to obtain the online time scheduling and power allocation based on the current causal knowledge of energy arrivals and channel fading at each time slot. Simulation results show that the proposed deep reinforcement learning based algorithm can achieve more than 90&#x0025; of maximum end-to-end throughput.

Xu Feng

Papers

Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing

Distributed Deep Learning-based Offloading for Mobile Edge Computing Networks

Multi-Server Multi-User Multi-Task Computation Offloading for Mobile Edge Computing Networks

Deep Reinforcement Learning-Based Task Offloading and Resource Allocation for Mobile Edge Computing

Deep RL-Based Time Scheduling and Power Allocation in EH Relay Communication Networks