Ubiquitous sensors and smart devices from factories and communities are generating massive amounts of data, and ever-increasing computing power is driving the core of computation and services from the cloud to the edge of the network. As an important enabler broadly changing people’s lives, from face recognition to ambitious smart factories and cities, developments of artificial intelligence (especially deep learning, DL) based applications and services are thriving. However, due to efficiency and latency issues, the current cloud computing service architecture hinders the vision of “providing artificial intelligence for every person and every organization at everywhere”. Thus, unleashing DL services using resources at the network edge near the data sources has emerged as a desirable solution. Therefore, edge intelligence , aiming to facilitate the deployment of DL services by edge computing, has received significant attention. In addition, DL, as the representative technique of artificial intelligence, can be integrated into edge computing frameworks to build intelligent edge for dynamic, adaptive edge maintenance and management. With regard to mutually beneficial edge intelligence and intelligent edge , this paper introduces and discusses: 1) the application scenarios of both; 2) the practical implementation methods and enabling technologies, namely DL training and inference in the customized edge computing framework; 3) challenges and future trends of more pervasive and fine-grained intelligence. We believe that by consolidating information scattered across the communication, networking, and DL areas, this survey can help readers to understand the connections between enabling technologies while promoting further discussions on the fusion of edge intelligence and intelligent edge , i.e., Edge DL.

Convergence of Edge Computing and Deep Learning: A Comprehensive Survey

Ubiquitous sensors and smart devices from factories and communities are generating massive amounts of data, and ever-increasing computing power is driving the core of computation and services from the cloud to the edge of the network. As an important enabler broadly changing people's lives, from face recognition to ambitious smart factories and cities, developments of artificial intelligence (especially deep learning, DL) based applications and services are thriving. However, due to efficiency and latency issues, the current cloud computing service architecture hinders the vision of "providing artificial intelligence for every person and every organization at everywhere". Thus, unleashing DL services using resources at the network edge near the data sources has emerged as a desirable solution. Therefore, edge intelligence, aiming to facilitate the deployment of DL services by edge computing, has received significant attention. In addition, DL, as the representative technique of artificial intelligence, can be integrated into edge computing frameworks to build intelligent edge for dynamic, adaptive edge maintenance and management. With regard to mutually beneficial edge intelligence and intelligent edge, this paper introduces and discusses: 1) the application scenarios of both; 2) the practical implementation methods and enabling technologies, namely DL training and inference in the customized edge computing framework; 3) challenges and future trends of more pervasive and fine-grained intelligence. We believe that by consolidating information scattered across the communication, networking, and DL areas, this survey can help readers to understand the connections between enabling technologies while promoting further discussions on the fusion of edge intelligence and intelligent edge, i.e., Edge DL.

Optimized container scheduling for data-intensive serverless edge computing

Many edge computing systems rely on virtual machines (VMs) to deliver their services. It is challenging, however, to deploy the virtualization mechanisms on edge computing hardware infrastructures. In this paper, we introduce the engineering and research trends of achieving efficient VM management in edge computing. We elaborate on: 1) the virtualization frameworks for edge computing developed in both the industry and the academia; 2) the virtualization techniques tailored for edge computing; 3) the placement and scheduling algorithms optimized for edge computing; and 4) the research problems in security related to virtualization of edge computing.

A Survey of Virtual Machine Management in Edge Computing

The Internet of Things (IoT) has created a ubiquitously connected world powered by a multitude of wired and wireless sensors generating a variety of heterogeneous data over time in a myriad of fields and applications. To extract complete information from these data, advanced artificial intelligence (AI) technology, especially deep learning (DL), has proved successful in facilitating data analytics, future prediction and decision making. The collective integration of AI and the IoT has greatly promoted the rapid development of AI-of-Things (AIoT) systems that analyze and respond to external stimuli more intelligently without involvement by humans. However, it is challenging or infeasible to process massive amounts of data in the cloud due to the destructive impact of the volume, velocity, and veracity of data and fatal transmission latency on networking infrastructures. These critical challenges can be adequately addressed by introducing edge computing. This article conducts an extensive survey of an end-edge-cloud orchestrated architecture for flexible AIoT systems. Specifically, it begins with articulating fundamental concepts including the IoT, AI and edge computing. Guided by these concepts, it explores the general AIoT architecture, presents a practical AIoT example to illustrate how AI can be applied in real-world applications and summarizes promising AIoT applications. Then, the emerging technologies for AI models regarding inference and training at the edge of the network are reviewed. Finally, the open challenges and future directions in this promising area are outlined.

A Survey of Recent Advances in Edge-Computing-Powered Artificial Intelligence of Things

In this paper, we introduce an infrastructure in edge computing environment, KubeEdge, to extend cloud capabilities to the edge. In the new form of cloud architecture, Cloud consists of computing resources both at centralized data centers and at distributed edges. KubeEdge infrastructure connects and coordinates two computing environments for applications leveraging both computing resources to achieve better performance and user experience. Technically, KubeEdge provides the network protocol infrastructure and the same runtime environment on the edge as in the cloud, which allows the seamless communication of applications with components running on edge nodes as well as cloud servers. It also allows the existing cloud services and cloud development model to be adopted at edge. Based on Kubernetes [1], KubeEdge architecture includes a network protocol stack called KubeBus, a distributed metadata store and synchronization service, and a lightweight agent (EdgeCore) for the edge. KubeBus is designed to have its own implementation of OSI network protocol layers, which connects servers at edge and VMs in the cloud as one virtual network. KubeBus provides a unified multitenant communication infrastructure with fault tolerance and high availability. The distributed metadata store and sync service is designed to support the offline scenario when edge nodes are not connected to the cloud. EdgeController component in KubeEdge architecture is a controller plugin for Kubernetes [1] to manage remote edge nodes and cloud VMs as one logical cluster, which enables KubeEdge to schedule, deploy and manage container applications across edge and cloud with the same API.

Extend Cloud to Edge with KubeEdge

Serverless computing is a popular cloud computing paradigm, which requires low response latency to handle on-demand user requests. There are two prominent techniques employed for reducing the response latency: keep fully initialized containers alive (Warm Container) or reduce the new container startup (cold start) latency. This paper presents the 3rd container startup mode: Hibernate Container, which starts faster than the cold start container mode and consumes less memory than the Warm Container mode. Hibernate Container is essentially a"deflated"Warm Container. Its application memory is swapped out to disk, the freed memory is reclaimed and file based mmap memory is cleaned-up. The Hibernate Container's deflated memory is inflated in response to user requests. As Hibernate Container's application is fully initialized, its response latency is less than the cold start mode; and as the application memory is deflated, its memory consumption is less than the Warm Container mode. Additionally, when a Hibernate Container is"woken up"to process a request, the Woken-up Container has similar response latency to Warm Container but less memory consumption because not all the deflated memory needs to be inflated. We implemented the Hibernate technique as part of the open source Quark secure container runtime project and our test demonstrated that Hibernate Container consumes about 7\% to 25\% of the Warm Container memory. All of this results in a higher deployment density, lower latency and appreciable improvements in the overall system performance.

Hibernate Container: A Deflated Container Mode for Fast Startup and High-density Deployment in Serverless Computing

Cloud-native containerized applications constantly seek high-performance and easy-to-operate container network solutions. RDMA network is a potential enabler with higher throughput and lower latency than the standard TCP/IP network stack. However, several challenges remain in equipping containerized applications with RDMA network: 1) How to deliver transparent improvements without modifying application code; 2) How to integrate RDMA-based network solutions with container orchestration systems; 3) How to efficiently utilize RDMA for container networks. In this paper, we present an RDMA-based container network solution, TCP Socket over RDMA (TSoR), which addresses all the above challenges. To transparently accelerate applications using POSIX socket interfaces without modifications, we integrate TSoR with a container runtime that can intercept system calls for socket interfaces. To be compatible with orchestration systems like Kubernetes, TSoR implements a container network following the Kubernetes network model and satisfies all requirements of the model. To leverage RDMA benefits, TSoR designs a high-performance network stack that efficiently transfers TCP traffic using RDMA network. Thus, TSoR provides a turn-key solution for existing Kubernetes clusters to adopt the high-performance RDMA network with minimal effort. Our evaluation results show that TSoR provides up to 2.3x higher throughput and 64\% lower latency for existing containerized applications, such as Redis key-value store and Node.js web server, with no code changes. TSoR code will be open-sourced.

Yulin Sun

Papers

Extend Cloud to Edge with KubeEdge

Hibernate Container: A Deflated Container Mode for Fast Startup and High-density Deployment in Serverless Computing

TSoR: TCP Socket over RDMA Container Network for Cloud Native Computing