Internet of Things (IoT) devices can apply mobile edge computing (MEC) and energy harvesting (EH) to provide high-level experiences for computational intensive applications and concurrently to prolong the lifetime of the battery. In this paper, we propose a reinforcement learning (RL) based offloading scheme for an IoT device with EH to select the edge device and the offloading rate according to the current battery level, the previous radio transmission rate to each edge device, and the predicted amount of the harvested energy. This scheme enables the IoT device to optimize the offloading policy without knowledge of the MEC model, the energy consumption model, and the computation latency model. Further, we present a deep RL-based offloading scheme to further accelerate the learning speed. Their performance bounds in terms of the energy consumption, computation latency, and utility are provided for three typical offloading scenarios and verified via simulations for an IoT device that uses wireless power transfer for energy harvesting. Simulation results show that the proposed RL-based offloading scheme reduces the energy consumption, computation latency, and task drop rate, and thus increases the utility of the IoT device in the dynamic MEC in comparison with the benchmark offloading schemes.

Learning-Based Computation Offloading for IoT Devices With Energy Harvesting

Wireless powered mobile-edge computing (MEC) has recently emerged as a promising paradigm to enhance the data processing capability of low-power networks, such as wireless sensor networks and internet of things (IoT). In this paper, we consider a wireless powered MEC network that adopts a binary offloading policy, so that each computation task of wireless devices (WDs) is either executed locally or fully offloaded to an MEC server. Our goal is to acquire an online algorithm that optimally adapts task offloading decisions and wireless resource allocations to the time-varying wireless channel conditions. This requires quickly solving hard combinatorial optimization problems within the channel coherence time, which is hardly achievable with conventional numerical optimization methods. To tackle this problem, we propose a Deep Reinforcement learning-based Online Offloading (DROO) framework that implements a deep neural network as a scalable solution that learns the binary offloading decisions from the experience. It eliminates the need of solving combinatorial optimization problems, and thus greatly reduces the computational complexity especially in large-size networks. To further reduce the complexity, we propose an adaptive procedure that automatically adjusts the parameters of the DROO algorithm on the fly. Numerical results show that the proposed algorithm can achieve near-optimal performance while significantly decreasing the computation time by more than an order of magnitude compared with existing optimization methods. For example, the CPU execution latency of DROO is less than 0.1 second in a 30-user network, making real-time and optimal offloading truly viable even in a fast fading environment.

Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks

Sending data to the cloud for analysis was a prominent trend during the past decades, driving cloud computing as a dominant computing paradigm. However, the dramatically increasing number of devices and data traffic in the Internet-of-Things (IoT) era are posing significant burdens on the capacity-limited Internet and uncontrollable service delay. It becomes difficult to meet the delay-sensitive and context-aware service requirements of IoT applications by using cloud computing alone. Facing these challenges, computing paradigms are shifting from the centralized cloud computing to distributed edge computing. Several new computing paradigms, including Transparent Computing, Mobile Edge Computing, Fog Computing, and Cloudlet, have emerged to leverage the distributed resources at network edge to provide timely and context-aware services. By integrating end devices, edge servers, and cloud, they form a hierarchical IoT architecture, i.e., End-Edge-Cloud orchestrated architecture to improve the performance of IoT systems. This article presents a comprehensive survey of these emerging computing paradigms from the perspective of end-edge-cloud orchestration. Specifically, we first introduce and compare the architectures and characteristics of different computing paradigms. Then, a comprehensive survey is presented to discuss state-of-the-art research in terms of computation offloading, caching, security, and privacy. Finally, some potential research directions are envisioned for fostering continuous research efforts.

A Survey on End-Edge-Cloud Orchestrated Network Computing Paradigms: Transparent Computing, Mobile Edge Computing, Fog Computing, and Cloudlet

Mobile edge computing (MEC) and wireless power transfer (WPT) are two promising techniques to enhance the computation capability and to prolong the operational time of low-power wireless devices that are ubiquitous in Internet of Things. However, the computation performance and the harvested energy are significantly impacted by the severe propagation loss. In order to address this issue, an unmanned aerial vehicle (UAV)-enabled MEC wireless powered system is studied in this paper. The computation rate maximization problems in a UAV-enabled MEC wireless powered system are investigated under both partial and binary computation offloading modes, subject to the energy harvesting causal constraint and the UAV's speed constraint. These problems are non-convex and challenging to solve. A two-stage algorithm and a three-stage alternative algorithm are respectively proposed for solving the formulated problems. The closed-form expressions for the optimal central processing unit frequencies, user offloading time, and user transmit power are derived. The optimal selection scheme on whether users choose to locally compute or offload computation tasks is proposed for the binary computation offloading mode. Simulation results show that our proposed resource allocation schemes outperforms other benchmark schemes. The results also demonstrate that the proposed schemes converge fast and have low computational complexity.

Computation Rate Maximization in UAV-Enabled Wireless Powered Mobile-Edge Computing Systems

To satisfy the increasing demand of mobile data traffic and meet the stringent requirements of the emerging Internet-of-Things (IoT) applications such as smart city, healthcare, and augmented/virtual reality (AR/VR), the fifth-generation (5G) enabling technologies are proposed and utilized in networks As an emerging key technology of 5G and a key enabler of IoT, multiaccess edge computing (MEC), which integrates telecommunication and IT services, offers cloud computing capabilities at the edge of the radio access network (RAN) By providing computational and storage resources at the edge, MEC can reduce latency for end users Hence, this article investigates MEC for 5G and IoT comprehensively It analyzes the main features of MEC in the context of 5G and IoT and presents several fundamental key technologies which enable MEC to be applied in 5G and IoT, such as cloud computing, software-defined networking/network function virtualization, information-centric networks, virtual machine (VM) and containers, smart devices, network slicing, and computation offloading In addition, this article provides an overview of the role of MEC in 5G and IoT, bringing light into the different MEC-enabled 5G and IoT applications as well as the promising future directions of integrating MEC with 5G and IoT Moreover, this article further elaborates research challenges and open issues of MEC for 5G and IoT Last but not least, we propose a use case that utilizes MEC to achieve edge intelligence in IoT scenarios

Toward Edge Intelligence: Multiaccess Edge Computing for 5G and Internet of Things

In this paper, we consider a multi-user mobile edge computing (MEC) network powered by wireless power transfer (WPT), where each energy-harvesting WD follows a binary computation offloading policy, i.e., data set of a task has to be executed as a whole either locally or remotely at the MEC server via task offloading. In particular, we are interested in maximizing the (weighted) sum computation rate of all the WDs in the network by jointly optimizing the individual computing mode selection (i.e., local computing or offloading) and the system transmission time allocation (on WPT and task offloading). The major difficulty lies in the combinatorial nature of multi-user computing mode selection and its strong coupling with transmission time allocation. To tackle this problem, we first consider a decoupled optimization, where we assume that the mode selection is given and propose a simple bi-section search algorithm to obtain the conditional optimal time allocation. On top of that, a coordinate descent method is devised to optimize the mode selection. The method is simple in implementation but may suffer from high computational complexity in a large-size network. To address this problem, we further propose a joint optimization method based on the ADMM (alternating direction method of multipliers) decomposition technique, which enjoys much slower increase of computational complexity as the networks size increases. Extensive simulations show that both the proposed methods can efficiently achieve near-optimal performance under various network setups, and significantly outperform the other representative benchmark methods considered.

Computation Rate Maximization for Wireless Powered Mobile-Edge Computing with Binary Computation Offloading

As an environment-friendly substitute for conventional fuel-powered vehicles, electric vehicles (EVs) and their components have been widely developed and deployed worldwide. The large-scale integration of EVs into power grid brings both challenges and opportunities to the system performance. On one hand, the load demand from EV charging imposes large impact on the stability and efficiency of power grid. On the other hand, EVs could potentially act as mobile energy storage systems to improve the power network performance, such as load flattening, fast frequency control, and facilitating renewable energy integration. Evidently, uncontrolled EV charging could lead to inefficient power network operation or even security issues. This spurs enormous research interests in designing charging coordination mechanisms. A key design challenge here lies in the lack of complete knowledge of events that occur in the future. Indeed, the amount of knowledge of future events significantly impacts the design of efficient charging control algorithms. This article focuses on introducing online EV charging scheduling techniques that deal with different degrees of uncertainty and randomness of future knowledge. Besides, we highlight the promising future research directions for EV charging control.

Online Charging Scheduling Algorithms of Electric Vehicles in Smart Grid: An Overview

We consider a two-level profit-maximizing strategy, including planning and control, for battery energy storage system (BESS) owners that participate in the primary frequency control (PFC) market. Specifically, the optimal BESS control minimizes the operating cost by keeping the state of charge (SoC) in an optimal range. Through rigorous analysis, we prove that the optimal BESS control is a "state-invariant" strategy in the sense that the optimal SoC range does not vary with the state of the system. As such, the optimal control strategy can be computed offline once and for all with very low complexity. Regarding the BESS planning, we prove that the the minimum operating cost is a decreasing convex function of the BESS energy capacity. This leads to the optimal BESS sizing that strikes a balance between the capital investment and operating cost. Our work here provides a useful theoretical framework for understanding the planning and control strategies that maximize the economic benefits of BESSs in ancillary service markets.

Profit-Maximizing Planning and Control of Battery Energy Storage Systems for Primary Frequency Control

The large-scale integration of plug-in electric vehicles (PEVs) to the power grid spurs the need for efficient charging coordination mechanisms. It can be shown that the optimal charging schedule smooths out the energy consumption over time so as to minimize the total energy cost. In practice, however, it is hard to smooth out the energy consumption perfectly, because the future PEV charging demand is unknown at the moment when the charging rate of an existing PEV needs to be determined. In this paper, we propose an Online cooRdinated CHARging Decision (ORCHARD) algorithm, which minimizes the energy cost without knowing the future information. Through rigorous proof, we show that ORCHARD is strictly feasible in the sense that it guarantees to fulfill all charging demands before due time. Meanwhile, it achieves the best known competitive ratio of 2.39. To further reduce the computational complexity of the algorithm, we propose a novel reduced-complexity algorithm to replace the standard convex optimization techniques used in ORCHARD. Through extensive simulations, we show that the average performance gap between ORCHARD and the offline optimal solution, which utilizes the complete future information, is as small as 14%. By setting a proper speeding factor, the average performance gap can be further reduced to less than 6%.

Online Coordinated Charging Decision Algorithm for Electric Vehicles without Future Information

The rapid emergence of electric vehicles (EVs) demands an advanced infrastructure of publicly accessible charging stations that provide efficient charging services. In this paper, we propose a new charging station operation mechanism, the JoAP, which jointly optimizes the EV admission control, pricing, and charging scheduling to maximize the charging station's profit. More specifically, by introducing a tandem queueing network model, we analytically characterize the average charging station profit as a function of the admission control and pricing policies. Based on the analysis, we characterize the optimal JoAP algorithm. Through extensive simulations, we demonstrate that the proposed JoAP algorithm on average can achieve 330% and 531% higher profit than a widely adopted benchmark method under two representative waiting-time penalty rates.

Ying Jun

Papers

Computation Rate Maximization for Wireless Powered Mobile-Edge Computing with Binary Computation Offloading

Online Charging Scheduling Algorithms of Electric Vehicles in Smart Grid: An Overview

Profit-Maximizing Planning and Control of Battery Energy Storage Systems for Primary Frequency Control

Online Coordinated Charging Decision Algorithm for Electric Vehicles without Future Information

Electrical Vehicle Charging Station Profit Maximization: Admission, Pricing, and Online Scheduling