Showing papers on "Resource management published in 2020"

Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks

Abstract: Unmanned aerial vehicles (UAVs) are capable of serving as aerial base stations (BSs) for providing both cost-effective and on-demand wireless communications. This article investigates dynamic resource allocation of multiple UAVs enabled communication networks with the goal of maximizing long-term rewards. More particularly, each UAV communicates with a ground user by automatically selecting its communicating user, power level and subchannel without any information exchange among UAVs. To model the dynamics and uncertainty in environments, we formulate the long-term resource allocation problem as a stochastic game for maximizing the expected rewards, where each UAV becomes a learning agent and each resource allocation solution corresponds to an action taken by the UAVs. Afterwards, we develop a multi-agent reinforcement learning (MARL) framework that each agent discovers its best strategy according to its local observations using learning. More specifically, we propose an agent-independent method, for which all agents conduct a decision algorithm independently but share a common structure based on Q-learning. Finally, simulation results reveal that: 1) appropriate parameters for exploitation and exploration are capable of enhancing the performance of the proposed MARL based resource allocation algorithm; 2) the proposed MARL algorithm provides acceptable performance compared to the case with complete information exchanges among UAVs. By doing so, it strikes a good tradeoff between performance gains and information exchange overheads.

Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider

Abstract: Function as a Service (FaaS) has been gaining popularity as a way to deploy computations to serverless backends in the cloud. This paradigm shifts the complexity of allocating and provisioning resources to the cloud provider, which has to provide the illusion of always-available resources (i.e., fast function invocations without cold starts) at the lowest possible resource cost. Doing so requires the provider to deeply understand the characteristics of the FaaS workload. Unfortunately, there has been little to no public information on these characteristics. Thus, in this paper, we first characterize the entire production FaaS workload of Azure Functions. We show for example that most functions are invoked very infrequently, but there is an 8-order-of-magnitude range of invocation frequencies. Using observations from our characterization, we then propose a practical resource management policy that significantly reduces the number of function cold starts, while spending fewer resources than state-of-the-practice policies.

Machine Learning for Resource Management in Cellular and IoT Networks: Potentials, Current Solutions, and Open Challenges

Abstract: Internet-of-Things (IoT) refers to a massively heterogeneous network formed through smart devices connected to the Internet. In the wake of disruptive IoT with a huge amount and variety of data, Machine Learning (ML) and Deep Learning (DL) mechanisms will play a pivotal role to bring intelligence to the IoT networks. Among other aspects, ML and DL can play an essential role in addressing the challenges of resource management in large-scale IoT networks. In this article, we conduct a systematic and in-depth survey of the ML- and DL-based resource management mechanisms in cellular wireless and IoT networks. We start with the challenges of resource management in cellular IoT and low-power IoT networks, review the traditional resource management mechanisms for IoT networks, and motivate the use of ML and DL techniques for resource management in these networks. Then, we provide a comprehensive survey of the existing ML- and DL-based resource management techniques in wireless IoT networks and the techniques specifically designed for HetNets, MIMO and D2D communications, and NOMA networks. To this end, we also identify the future research directions in using ML and DL for resource allocation and management in IoT networks.

Blockchain and Machine Learning for Communications and Networking Systems

Abstract: Recently, with the rapid development of information and communication technologies, the infrastructures, resources, end devices, and applications in communications and networking systems are becoming much more complex and heterogeneous. In addition, the large volume of data and massive end devices may bring serious security, privacy, services provisioning, and network management challenges. In order to achieve decentralized, secure, intelligent, and efficient network operation and management, the joint consideration of blockchain and machine learning (ML) may bring significant benefits and have attracted great interests from both academia and industry. On one hand, blockchain can significantly facilitate training data and ML model sharing, decentralized intelligence, security, privacy, and trusted decision-making of ML. On the other hand, ML will have significant impacts on the development of blockchain in communications and networking systems, including energy and resource efficiency, scalability, security, privacy, and intelligent smart contracts. However, some essential open issues and challenges that remain to be addressed before the widespread deployment of the integration of blockchain and ML, including resource management, data processing, scalable operation, and security issues. In this paper, we present a survey on the existing works for blockchain and ML technologies. We identify several important aspects of integrating blockchain and ML, including overview, benefits, and applications. Then we discuss some open issues, challenges, and broader perspectives that need to be addressed to jointly consider blockchain and ML for communications and networking systems.

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Abstract: It has been a long-held belief that judicious resource allocation is critical to mitigating interference, improving network efficiency, and ultimately optimizing wireless communication performance. The traditional wisdom is to explicitly formulate resource allocation as an optimization problem and then exploit mathematical programming to solve the problem to a certain level of optimality. Nonetheless, as wireless networks become increasingly diverse and complex, for example, in the high-mobility vehicular networks, the current design methodologies face significant challenges and thus call for rethinking of the traditional design philosophy. Meanwhile, deep learning, with many success stories in various disciplines, represents a promising alternative due to its remarkable power to leverage data for problem solving. In this article, we discuss the key motivations and roadblocks of using deep learning for wireless resource allocation with application to vehicular networks. We review major recent studies that mobilize the deep-learning philosophy in wireless resource allocation and achieve impressive results. We first discuss deep-learning-assisted optimization for resource allocation. We then highlight the deep reinforcement learning approach to address resource allocation problems that are difficult to handle in the traditional optimization framework. We also identify some research directions that deserve further investigation.

HFEL: Joint Edge Association and Resource Allocation for Cost-Efficient Hierarchical Federated Edge Learning

Abstract: Federated Learning (FL) has been proposed as an appealing approach to handle data privacy issue of mobile devices compared to conventional machine learning at the remote cloud with raw user data uploading By leveraging edge servers as intermediaries to perform partial model aggregation in proximity and relieve core network transmission overhead, it enables great potentials in low-latency and energy-efficient FL Hence we introduce a novel Hierarchical Federated Edge Learning (HFEL) framework in which model aggregation is partially migrated to edge servers from the cloud We further formulate a joint computation and communication resource allocation and edge association problem for device users under HFEL framework to achieve global cost minimization To solve the problem, we propose an efficient resource scheduling algorithm in the HFEL framework It can be decomposed into two subproblems: resource allocation given a scheduled set of devices for each edge server and edge association of device users across all the edge servers With the optimal policy of the convex resource allocation subproblem for a set of devices under a single edge server, an efficient edge association strategy can be achieved through iterative global cost reduction adjustment process, which is shown to converge to a stable system point Extensive performance evaluations demonstrate that our HFEL framework outperforms the proposed benchmarks in global cost saving and achieves better training performance compared to conventional federated learning

Using Landsat satellite data for assessing the land use and land cover change in Kashmir valley

Abstract: Land use and land cover (LULC) change has been one of the most immense and perceptible transformations of the earth’s surface. Evaluating LULC change at varied spatial scales is imperative in wide range of perspectives such as environmental conservation, resource management, land use planning, and sustainable development. This work aims to examine the land use and land cover changes in the Kashmir valley between the time periods from 1992–2001–2015 using a set of compatible moderate resolution Landsat satellite imageries. Supervised approach with maximum likelihood classifier was adopted for the classification and generation of LULC maps for the selected time periods. Results reveal that there have been substantial changes in the land use and cover during the chosen time periods. In general, three land use and land cover change patterns were observed in the study area: (1) consistent increase of the area under marshy, built-up, barren, plantation, and shrubs; (2) continuous decrease in agriculture and water; (3) decrease (1992–2001) and increase (2001–2015) in forest and pasture classes. In terms of the area under each LULC category, most significant changes have been observed in agriculture (−), plantation (+), built-up (+), and water (−); however, with reference to percent change within each class, the maximum variability was recorded in built-up (198.45%), plantation (87.98%), pasture (− 71%), water (− 48%) and agriculture (− 28.85%). The massive land transformation is largely driven by anthropogenic actions and has been mostly adverse in nature, giving rise to multiple environmental issues in the ecologically sensitive Kashmir valley.

Joint Optimization of Offloading Utility and Privacy for Edge Computing Enabled IoT

Abstract: Currently, edge computing (EC), emerging as a burgeoning paradigm, is powerful in handling real-time resource provision for Internet of Things (IoT) applications. However, due to the spatial distribution of geographically sparse IoT devices and the resource limitations of EC units (ECUs), the resource utilization of corresponding edge servers is relatively insufficient and the execution performance is ineffective to some extent. A privacy leakage, including personal information, location, media data, etc., during the transmission process from IoT devices to edge servers severely restricts the application of ECUs in IoT. To address these challenges, a two-phase offloading optimization strategy is put forward for joint optimization of offloading utility and privacy in EC enabled IoT. Technically, a utility-aware task offloading method, named UTO, is devised first to obtain the goal of maximizing the resource utilization of ECUs and minimizing the implementation time cost. Then a joint optimization method, named JOM, for utility and privacy tradeoffs is designed to balance the privacy preservation and execution performance. Eventually, the experimental evaluations are designed to illustrate the efficiency and reliability of UTO and JOM.

Efficient Computing Resource Sharing for Mobile Edge-Cloud Computing Networks

Abstract: Both the edge and the cloud can provide computing services for mobile devices to enhance their performance. The edge can reduce the conveying delay by providing local computing services while the cloud can support enormous computing requirements. Their cooperation can improve the utilization of computing resources and ensure the QoS, and thus is critical to edge-cloud computing business models. This paper proposes an efficient framework for mobile edge-cloud computing networks, which enables the edge and the cloud to share their computing resources in the form of wholesale and buyback. To optimize the computing resource sharing process, we formulate the computing resource management problems for the edge servers to manage their wholesale and buyback scheme and the cloud to determine the wholesale price and its local computing resources. Then, we solve these problems from two perspectives: i) social welfare maximization and ii) profit maximization for the edge and the cloud. For i), we have proved the concavity of the social welfare and proposed an optimal cloud computing resource management to maximize the social welfare. For ii), since it is difficult to directly prove the convexity of the primal problem, we first proved the concavity of the wholesaled computing resources with respect to the wholesale price and designed an optimal pricing and cloud computing resource management to maximize their profits. Numerical evaluations show that the total profit can be maximized by social welfare maximization while the respective profits can be maximized by the optimal pricing and cloud computing resource management.

Learning-Based Energy-Efficient Resource Management by Heterogeneous RF/VLC for Ultra-Reliable Low-Latency Industrial IoT Networks

Abstract: Smart factory under Industry 4.0 and industrial Internet of Things (IoT) has attracted much attention from both academia and industry. In wireless industrial networks, industrial IoT and IoT devices have different quality-of-service (QoS) requirements, ranging from ultra-reliable low-latency communications (URLLC) to high transmission data rates. These industrial networks will be highly complex and heterogeneous, as well as the spectrum and energy resources are severely limited. Hence, this article presents a heterogeneous radio frequency (RF)/visible light communication (VLC) industrial network architecture to guarantee the different QoS requirements, where RF is capable of offering wide-area coverage and VLC has the ability to provide high transmission data rate. A joint uplink and downlink energy-efficient resource management decision-making problem (network selection, subchannel assignment, and power management) is formulated as a Markov decision process. In addition, a new deep post-decision state (PDS)-based experience replay and transfer (PDS-ERT) reinforcement learning algorithm is proposed to learn the optimal policy. Simulation results corroborate the superiority in performance of the presented heterogeneous network, and verify that the proposed PDS-ERT learning algorithm outperforms other existing algorithms in terms of meeting the energy efficiency and the QoS requirements.

Deep Reinforcement Learning Based Resource Management for Multi-Access Edge Computing in Vehicular Networks

Abstract: In this paper, we study joint allocation of the spectrum, computing, and storing resources in a multi-access edge computing (MEC)-based vehicular network. To support different vehicular applications, we consider two typical MEC architectures and formulate multi-dimensional resource optimization problems accordingly, which are usually with high computation complexity and overlong problem-solving time. Thus, we exploit reinforcement learning (RL) to transform the two formulated problems and solve them by leveraging the deep deterministic policy gradient (DDPG) and hierarchical learning architectures. Via off-line training, the network dynamics can be automatically learned and appropriate resource allocation decisions can be rapidly obtained to satisfy the quality-of-service (QoS) requirements of vehicular applications. From simulation results, the proposed resource management schemes can achieve high delay/QoS satisfaction ratios.

Dynamic Task Offloading and Resource Allocation for Mobile-Edge Computing in Dense Cloud RAN

Abstract: With the unprecedented development of smart mobile devices (SMDs), e.g., Internet-of-Things devices and smartphones, various computation-intensive applications are explosively increasing in ultradense networks (UDNs). Mobile-edge computing (MEC) has emerged as a key technology to alleviate the computation workloads of SMDs and decrease service latency for computation-intensive applications. With the benefits of network function virtualization, MEC can be integrated with the cloud radio access network (C-RAN) in UDNs for computation and communication cooperation. However, with stochastic computation task arrivals and time-varying channel states, it is challenging to offload computation tasks online with energy-efficient computation and radio resource management. In this article, we investigate the task offloading and resource allocation problem in MEC-enabled dense C-RAN, aiming at optimizing network energy efficiency. A stochastic mixed-integer nonlinear programming problem is formulated to jointly optimize the task offloading decision, elastic computation resource scheduling, and radio resource allocation. To tackle the problem, the Lyapunov optimization theory is introduced to decompose the original problem into four individual subproblems which are solved by convex decomposition methods and matching game. We theoretically analyze the tradeoff between energy efficiency and service delay. Extensive simulations evaluate the impacts of system parameters on both energy efficiency and service delay. The simulation results also validate the superiority of the proposed task offloading and resource allocation scheme in dense C-RAN.

Examining sustainable supply chain management of SMEs using resource based view and institutional theory

Abstract: The long-term viability of an organization hinges on social, environmental, and economic measures. However, based on extensive review of the literature, we have observed that measuring and improving the sustainable performance of supply chains is complex. We have grounded our theoretical framework in institutional theory and resource-based view and drawn thirteen hypotheses. We developed our instrument scientifically to validate our model and test our research hypotheses. The data was collected from the Indian auto components industry following Dillman’s total design test method. We gathered 205 usable responses. Following Peng and Lai’s (J Oper Manag 30(6):467–480, 2012) arguments, we have tested our model using variance-based structural equation modeling (PLS-SEM). We found that the constructs used for building our theoretical model possess construct validity and further satisfy the specified criteria for goodness of fit. The hypotheses test further suggests that coercive pressures under the mediation effect of top management belief and participation have significant influence on resource selection (i.e. supply chain connectivity and supply chain information sharing). The supply chain connectivity and supply chain information sharing have significant influence on environmental performance. Contrary to our belief, the normative and mimetic pressures have no significant influence on top management participation. The managerial implications of the findings are also discussed.

Latency Minimization for D2D-Enabled Partial Computation Offloading in Mobile Edge Computing

Abstract: We consider Device-to-Device (D2D)-enabled mobile edge computing offloading scenario, where a device can partially offload its computation task to the edge server or exploit the computation resources of proximal devices. Keeping in view the millisecond-scale latency requirement in 5G service scenarios and the spectrum scarcity, we focus on minimizing the sum of task execution latency of all the devices in a shared spectrum with interference. In particular, we provide an integrated framework for partial offloading and interference management using orthogonal frequency-division multiple access (OFDMA) scheme. Accordingly, we formulate total latency minimization as a mixed integer nonlinear programming (MINLP) problem by considering desired energy consumption, partial offloading, and resource allocation constraints. We use decomposition approach to solve our problem and propose a novel scheme named Joint Partial Offloading and Resource Allocation (JPORA). With aim to reduce the task execution latency, JPORA iteratively adjusts data segmentation and solves the underlying problem of quality of service (QoS)-aware communication resource allocation to the cellular links, and interference-aware communication resource allocation to D2D links. Extensive evaluation results demonstrate that JPORA achieves the lowest latency as compared to the other baseline schemes, meanwhile limiting the local energy consumption of user devices.

Adaptive Resource Allocation in Future Wireless Networks With Blockchain and Mobile Edge Computing

Abstract: In this paper, we present a blockchain-based mobile edge computing (B-MEC) framework for adaptive resource allocation and computation offloading in future wireless networks, where the blockchain works as an overlaid system to provide management and control functions. In this framework, how to reach a consensus between the nodes while simultaneously guaranteeing the performance of both MEC and blockchain systems is a major challenge. Meanwhile, resource allocation, block size, and the number of consecutive blocks produced by each producer are critical to the performance of B-MEC. Therefore, an adaptive resource allocation and block generation scheme is proposed. To improve the throughput of the overlaid blockchain system and the quality of services (QoS) of the users in the underlaid MEC system, spectrum allocation, size of the blocks, and number of producing blocks for each producer are formulated as a joint optimization problem, where the time-varying wireless links and computation capacity of the MEC servers are considered. Since this problem is intractable using traditional methods, we resort to the deep reinforcement learning approach. Simulation results show the effectiveness of the proposed approach by comparing with other baseline methods.

Dynamical Resource Allocation in Edge for Trustable Internet-of-Things Systems: A Reinforcement Learning Method

Abstract: Edge computing (EC) is now emerging as a key paradigm to handle the increasing Internet-of-Things (IoT) devices connected to the edge of the network. By using the services deployed on the service provisioning system which is made up of edge servers nearby, these IoT devices are enabled to fulfill complex tasks effectively. Nevertheless, it also brings challenges in trustworthiness management. The volatile environment will make it difficult to comply with the service-level agreement (SLA), which is an important index of trustworthiness declared by these IoT services. In this article, by denoting the trustworthiness gain with how well the SLA can comply, we first encode the state of the service provisioning system and the resource allocation scheme and model the adjustment of allocated resources for services as a Markov decision process (MDP). Based on these, we get a trained resource allocating policy with the help of the reinforcement learning (RL) method. The trained policy can always maximize the services’ trustworthiness gain by generating appropriate resource allocation schemes dynamically according to the system states. By conducting a series of experiments on the YouTube request dataset, we show that the edge service provisioning system using our approach has 21.72% better performance at least compared to baselines.

GAN-Powered Deep Distributional Reinforcement Learning for Resource Management in Network Slicing

Abstract: Network slicing is a key technology in 5G communications system. Its purpose is to dynamically and efficiently allocate resources for diversified services with distinct requirements over a common underlying physical infrastructure. Therein, demand-aware resource allocation is of significant importance to network slicing. In this paper, we consider a scenario that contains several slices in a radio access network with base stations that share the same physical resources (e.g., bandwidth or slots). We leverage deep reinforcement learning (DRL) to solve this problem by considering the varying service demands as the environment state and the allocated resources as the environment action . In order to reduce the effects of the annoying randomness and noise embedded in the received service level agreement (SLA) satisfaction ratio (SSR) and spectrum efficiency (SE), we primarily propose generative adversarial network-powered deep distributional Q network (GAN-DDQN) to learn the action-value distribution driven by minimizing the discrepancy between the estimated action-value distribution and the target action-value distribution. We put forward a reward-clipping mechanism to stabilize GAN-DDQN training against the effects of widely-spanning utility values. Moreover, we further develop Dueling GAN-DDQN, which uses a specially designed dueling generator, to learn the action-value distribution by estimating the state-value distribution and the action advantage function. Finally, we verify the performance of the proposed GAN-DDQN and Dueling GAN-DDQN algorithms through extensive simulations.

Blockchain Meets Edge Computing: Stackelberg Game and Double Auction Based Task Offloading for Mobile Blockchain

Abstract: Blockchain technology is developing rapidly and has been applied in various aspects, among which there are broad prospects in Internet of Things (IoT). However, IoT mobile devices are restricted in communication and computation due to mobility and portability, so that they can’t afford the high computing cost for blockchain mining process. To solve it, the free resources displayed on non-mining-devices and edge cloud are selected to construct collaborative mining network(CMN) to execute mining tasks for mobile blockchain. Miners can offload their mining tasks to non-mining-devices within a CMN or the edge cloud when there are insufficient resources. Considering competition for resource of non-mining-devices, resource allocation problem in a CMN is formulated as a double auction game, among which Bayes-Nash Equilibrium (BNE) is analyzed to figure out the optimal auction price. When offloading to edge cloud, Stackelberg game is adopted to model interactions between edge cloud operator and different CMNs to obtain the optimal resource price and devices’ resource demands. The mechanism realizes improving the mining utility in mining networks while ensuring the maximum profit of edge cloud operator. Finally, profits of mining networks are compared with an existing mode which only considers offloading to edge cloud. Under the proposed mechanism, mining networks obtain 6.86% more profits on average.

Deep Reinforcement Learning for Dynamic Uplink/Downlink Resource Allocation in High Mobility 5G HetNet

Abstract: Recently, the 5G is widely deployed for supporting communications of high mobility nodes including train, vehicular and unmanned aerial vehicles (UAVs) largely emerged as the main components for constructing the wireless heterogeneous network (HetNet). To further improve the radio utilization, the Time Division Duplex (TDD) is considered to be the potential full-duplex communication technology in the high mobility 5G network. However, the high mobility of users leads to the high dynamic network traffic and unpredicted link state change. A new method to predict the dynamic traffic and channel condition and schedule the TDD configuration in real-time is essential for the high mobility environment. In this paper, we investigate the channel model in the high mobility and heterogeneous network and proposed a novel deep reinforcement learning based intelligent TDD configuration algorithm to dynamically allocate radio resources in an online manner. In the proposal, the deep neural network is employed to extract the features of the complex network information, and the dynamic Q-value iteration based reinforcement learning with experience replay memory mechanism is proposed to adaptively change TDD Up/Down-link ratio by evaluated rewards. The simulation results show that the proposal achieves significant network performance improvement in terms of both network throughput and packet loss rate, comparing with conventional TDD resource allocation algorithms.

Deep-Reinforcement-Learning-Based Mode Selection and Resource Allocation for Cellular V2X Communications

Abstract: Cellular vehicle-to-everything (V2X) communication is crucial to support future diverse vehicular applications. However, for safety-critical applications, unstable vehicle-to-vehicle (V2V) links, and high signaling overhead of centralized resource allocation approaches become bottlenecks. In this article, we investigate a joint optimization problem of transmission mode selection and resource allocation for cellular V2X communications. In particular, the problem is formulated as a Markov decision process, and a deep reinforcement learning (DRL)-based decentralized algorithm is proposed to maximize the sum capacity of vehicle-to-infrastructure users while meeting the latency and reliability requirements of V2V pairs. Moreover, considering training limitation of local DRL models, a two-timescale federated DRL algorithm is developed to help obtain robust models. Wherein, the graph theory-based vehicle clustering algorithm is executed on a large timescale and in turn, the federated learning algorithm is conducted on a small timescale. The simulation results show that the proposed DRL-based algorithm outperforms other decentralized baselines, and validate the superiority of the two-timescale federated DRL algorithm for newly activated V2V pairs.

Performance Evaluation Metrics for Cloud, Fog and Edge Computing: A Review, Taxonomy, Benchmarks and Standards for Future Research

Abstract: Optimization is an inseparable part of Cloud computing, particularly with the emergence of Fog and Edge paradigms. Not only these emerging paradigms demand reevaluating cloud-native optimizations and exploring Fog and Edge-based solutions, but also the objectives require significant shift from considering only latency to energy, security, reliability and cost. Hence, it is apparent that optimization objectives have become diverse and lately Internet of Things (IoT)-specific born objectives must come into play. This is critical as incorrect selection of metrics can mislead the developer about the real performance. For instance, a latency-aware auto-scaler must be evaluated through latency-related metrics as response time or tail latency; otherwise the resource manager is not carefully evaluated even if it can reduce the cost. Given such challenges, researchers and developers are struggling to explore and utilize the right metrics to evaluate the performance of optimization techniques such as task scheduling, resource provisioning, resource allocation, resource scheduling and resource execution. This is challenging due to (1) novel and multi-layered computing paradigm, e.g., Cloud, Fog and Edge, (2) IoT applications with different requirements, e.g., latency or privacy, and (3) not having a benchmark and standard for the evaluation metrics. In this paper, by exploring the literature, (1) we present a taxonomy of the various real-world metrics to evaluate the performance of cloud, fog, and edge computing; (2) we survey the literature to recognize common metrics and their applications; and (3) outline open issues for future research. This comprehensive benchmark study can significantly assist developers and researchers to evaluate performance under realistic metrics and standards to ensure their objectives will be achieved in the production environments.

Cooperative Offloading and Resource Management for UAV-Enabled Mobile Edge Computing in Power IoT System

Abstract: The lack of the computation services in remote areas motivates power Internet of Things (IoT) to apply unmanned aerial vehicle (UAV)-enabled mobile edge computing (MEC) technology However, the computation services will be significantly affected by the UAVs’ capacities, and distinct power IoT applications In this paper, we firstly propose a cooperative UAV-enabled MEC network structure in which the UAVs are able to help other UAVs to execute the computation tasks Then, a cooperative computation offloading scheme is presented while considering the interference mitigation from UAVs to devices To maximize the long-term utility of the proposed UAV-enabled MEC network, an optimization problem is formulated to obtain the optimal computation offloading decisions, and resource management policies Considering the random devices’ demands and time-varying communication channels, the problem is further formulated as a semi-Markov process, and the deep reinforcement learning based algorithms are proposed in both of the centralized and distributed UAV-enabled MEC networks Finally, we evaluate the performance of the proposed DRL-based schemes in the UAV-enabled MEC framework by giving numerical results

Deep-Learning-Based Joint Resource Scheduling Algorithms for Hybrid MEC Networks

Abstract: In this article, we consider a hybrid mobile edge computing (H-MEC) platform, which includes ground stations (GSs), ground vehicles (GVs), and unmanned aerial vehicles (UAVs), all with the mobile edge cloud installed to enable user equipments (UEs) or Internet of Things (IoT) devices with intensive computing tasks to offload. Our objective is to obtain an online offloading algorithm to minimize the energy consumption of all the UEs, by jointly optimizing the positions of GVs and UAVs, user association and resource allocation in real time, while considering the dynamic environment. To this end, we propose a hybrid deep-learning-based online offloading (H2O) framework where a large-scale path-loss fuzzy c-means (LS-FCM) algorithm is first proposed and used to predict the optimal positions of GVs and UAVs. Second, a fuzzy membership matrix U-based particle swarm optimization (U-PSO) algorithm is applied to solve the mixed-integer nonlinear programming (MINLP) problems and generate the sample data sets for the deep neural network (DNN) where the fuzzy membership matrix can capture the small-scale fading effects and the information of mutual interference. Third, a DNN with the scheduling layer is introduced to provide the user association and computing resource allocation under the practical latency requirement of the tasks and limited available computing resource of H-MEC. In addition, different from the traditional DNN predictor, we only input one UE’s information to the DNN at one time, which will be suitable for the scenarios where the number of UE is varying and avoid the curse of dimensionality in DNN.

Resource Allocation for Vehicular Fog Computing Using Reinforcement Learning Combined With Heuristic Information

Abstract: Internet of Vehicles (IoV) has emerged as a key component of smart cities. Connected vehicles are increasingly processing real-time data to respond immediately to user requests. However, the data must be sent to a remote cloud for processing. To resolve this issue, vehicular fog computing (VFC) has emerged as a promising paradigm that improves the quality of computation experiences for vehicles by offloading computation tasks from the cloud to network edges. Nevertheless, due to the resource restrictions of fog computing, only a limited number of vehicles are able to use it while it is still challenging to provide real-time responses for vehicular applications, such as traffic and accident warnings in the highly dynamic IoV environment. Therefore, in this article, we formulate the problem of allocating the limited fog resources to vehicular applications such that the service latency is minimized, by utilizing parked vehicles. We then propose a heuristic algorithm to efficiently find the solutions of the problem formulation. In addition, the proposed algorithm is combined with reinforcement learning to make more efficient resource allocation decisions, leveraging the vehicles’ movement and parking status collected from the smart environment of the city. Our simulation results show that our VFC resource allocation algorithm can achieve higher service satisfaction compared to conventional resource allocation algorithms.

FIRM: An intelligent fine-grained resource management framework for SLO-oriented microservices

Abstract: Modern user-facing latency-sensitive web services include numerous distributed, intercommunicating microservices that promise to simplify software development and operation. However, multiplexing of compute resources across microservices is still challenging in production because contention for shared resources can cause latency spikes that violate the service-level objectives (SLOs) of user requests. This paper presents FIRM, an intelligent fine-grained resource management framework for predictable sharing of resources across microservices to drive up overall utilization. FIRM leverages online telemetry data and machine-learning methods to adaptively (a) detect/localize microservices that cause SLO violations, (b) identify low-level resources in contention, and (c) take actions to mitigate SLO violations via dynamic reprovisioning. Experiments across four microservice benchmarks demonstrate that FIRM reduces SLO violations by up to 16x while reducing the overall requested CPU limit by up to 62%. Moreover, FIRM improves performance predictability by reducing tail latencies by up to 11x.

Distributed Dynamic Resource Management and Pricing in the IoT Systems With Blockchain-as-a-Service and UAV-Enabled Mobile Edge Computing

Abstract: In this article, we study the pricing and resource management in the Internet of Things (IoT) system with blockchain-as-a-service (BaaS) and mobile-edge computing (MEC). The BaaS model includes the cloud-based server to perform blockchain tasks and the set of peers to collect data from local IoT devices. The MEC model consists of the set of terrestrial and aerial base stations (BSs), i.e., unmanned aerial vehicles (UAVs), to forward the tasks of peers to the BaaS server. Each BS is also equipped with an MEC server to run some blockchain tasks. As the BSs can be privately owned or controlled by different operators, there is no information exchange among them. We show that the resource management and pricing in the BaaS-MEC system are modeled as a stochastic Stackelberg game with multiple leaders and incomplete information about actions of leaders/BSs and followers/peers. We formulate a novel hierarchical reinforcement learning (RL) algorithm for the decision makings of BSs and peers. We also develop an unsupervised hierarchical deep learning (HDL) algorithm that combines deep $Q$ -learning (DQL) for BSs with the Bayesian deep learning (BDL) for peers. We prove that the proposed algorithms converge to stable states in which the peers’ actions are the best responses to optimal actions of BSs.

When Deep Reinforcement Learning Meets Federated Learning: Intelligent Multi-Timescale Resource Management for Multi-access Edge Computing in 5G Ultra Dense Network

Abstract: Ultra-dense edge computing (UDEC) has great potential, especially in the 5G era, but it still faces challenges in its current solutions, such as the lack of: i) efficient utilization of multiple 5G resources (e.g., computation, communication, storage and service resources); ii) low overhead offloading decision making and resource allocation strategies; and iii) privacy and security protection schemes. Thus, we first propose an intelligent ultra-dense edge computing (I-UDEC) framework, which integrates blockchain and Artificial Intelligence (AI) into 5G ultra-dense edge computing networks. First, we show the architecture of the framework. Then, in order to achieve real-time and low overhead computation offloading decisions and resource allocation strategies, we design a novel two-timescale deep reinforcement learning (\textit{2Ts-DRL}) approach, consisting of a fast-timescale and a slow-timescale learning process, respectively. The primary objective is to minimize the total offloading delay and network resource usage by jointly optimizing computation offloading, resource allocation and service caching placement. We also leverage federated learning (FL) to train the \textit{2Ts-DRL} model in a distributed manner, aiming to protect the edge devices' data privacy. Simulation results corroborate the effectiveness of both the \textit{2Ts-DRL} and FL in the I-UDEC framework and prove that our proposed algorithm can reduce task execution time up to 31.87%.

A Smart Waste Management Solution Geared towards Citizens.

Abstract: Global industry is undergoing major transformations with the genesis of a new paradigm known as the Internet of Things (IoT) with its underlying technologies. Many company leaders are investing more effort and money in transforming their services to capitalize on the benefits provided by the IoT. Thereby, the decision makers in public waste management do not want to be outdone, and it is challenging to provide an efficient and real-time waste management system. This paper proposes a solution (hardware, software, and communications) that aims to optimize waste management and include a citizen in the process. The system follows an IoT-based approach where the discarded waste from the smart bin is continuously monitored by sensors that inform the filling level of each compartment, in real-time. These data are stored and processed in an IoT middleware providing information for collection with optimized routes and generating important statistical data for monitoring the waste collection accurately in terms of resource management and the provided services for the community. Citizens can easily access information about the public waste bins through the Web or a mobile application. The creation of the real prototype of the smart container, the development of the waste management application and a real-scale experiment use case for evaluation, demonstration, and validation show that the proposed system can efficiently change the way people deal with their garbage and optimize economic and material resources.