In this paper, we propose a Distributed Intelligent Video Surveillance (DIVS) system using Deep Learning (DL) algorithms and deploy it in an edge computing environment. We establish a multi-layer edge computing architecture and a distributed DL training model for the DIVS system. The DIVS system can migrate computing workloads from the network center to network edges to reduce huge network communication overhead and provide low-latency and accurate video analysis solutions. We implement the proposed DIVS system and address the problems of parallel training, model synchronization, and workload balancing. Task-level parallel and model-level parallel training methods are proposed to further accelerate the video analysis process. In addition, we propose a model parameter updating method to achieve model synchronization of the global DL model in a distributed EC environment. Moreover, a dynamic data migration approach is proposed to address the imbalance of workload and computational power of edge nodes. Experimental results showed that the EC architecture can provide elastic and scalable computing power, and the proposed DIVS system can efficiently handle video surveillance and analysis tasks.

Distributed Deep Learning Model for Intelligent Video Surveillance Systems with Edge Computing

We investigate the security of Split Learning -- a novel collaborative machine learning framework that enables peak performance by requiring minimal resources consumption. In the present paper, we expose vulnerabilities of the protocol and demonstrate its inherent insecurity by introducing general attack strategies targeting the reconstruction of clients' private training sets. More prominently, we show that a malicious server can actively hijack the learning process of the distributed model and bring it into an insecure state that enables inference attacks on clients' data. We implement different adaptations of the attack and test them on various datasets as well as within realistic threat scenarios. We demonstrate that our attack is able to overcome recently proposed defensive techniques aimed at enhancing the security of the split learning protocol. Finally, we also illustrate the protocol's insecurity against malicious clients by extending previously devised attacks for Federated Learning. To make our results reproducible, we made our code available at this https URL.

Unleashing the Tiger: Inference Attacks on Split Learning

The existing satellite-terrestrial integrated networks (STINs) suffer from security and privacy concerns due to the limited resources, poor attack resistance and high privacy requirements of satellite networks. Network Intrusion Detection System (NIDS) is intended to provide a high level of protection for modern network environments, but how to implement distributed NIDS on STINs has not been widely discussed. At the same time, satellite networks have always lacked real and effective security data sets as references. To solve these problems, we propose a distributed NIDS using Federal Learning (FL) in STIN to properly allocate resources in each domain to analyze and block malicious traffic, especially distributed denial-of-service (DDoS) attacks. Specifically, we first design a typical STIN topology, on the basis of which we collect and design security data sets adapted to satellite and terrestrial networks in STIN, respectively. To address the problem of poor attack resistance of satellite networks, we propose a satellite network topology optimization algorithm to reduce the difficulty in tracing malicious packets due to frequent link switching. In order to solve the problem of limited resources and high privacy requirements of satellite networks, we propose an algorithm for FL adaptation to STIN, and build a distributed NIDS using FL in STIN. Finally, we deploy the designed distributed NIDS in a prototype system and evaluate our proposed distributed NIDS with a large number of simulations of randomly generated malicious traffic. Related results demonstrate that the performance of our approach is better than traditional deep learning and intrusion detection methods in terms of malicious traffic recognition rate, packet loss rate, and CPU utilization.

/pdf/distributed-network-intrusion-detection-system-in-satellite-54n72n92pw.pdf

Distributed Network Intrusion Detection System in Satellite-Terrestrial Integrated Networks Using Federated Learning

Deep learning methods for solving linear inverse problems: Research directions and paradigms

With the increasing number of vehicles, the traffic congestion is becoming more and more serious. In order to alleviate such a problem, this article considers transmission and inference delay of cloud centralized computing in the software defined Internet of Vehicles (SDIoV), and builds a new SDIoV architecture based on edge intelligence, for supporting real-time vehicle routing decision through distributed multi-agent reinforcement learning model. Then, a software defined device collaboration optimization method is designed to improve the efficiency of distributed training. Combined with multi-agent reinforcement learning, a distributed-learning-based vehicle routing decision algorithm (DLRD) is proposed to adaptively adjust vehicle routing online. The performed simulations show that the DLRD can successfully realize real-time routing decision for vehicles and alleviate traffic congestion with the dynamic changes of the road environment.

Distributed Learning for Vehicle Routing Decision in Software Defined Internet of Vehicles

Distributed deep learning systems (DDLS) train deep neural network models by utilizing the distributed resources of a cluster. Developers of DDLS are required to make many decisions to process their particular workloads in their chosen environment efficiently. The advent of GPU-based deep learning, the ever-increasing size of datasets, and deep neural network models, in combination with the bandwidth constraints that exist in cluster environments require developers of DDLS to be innovative in order to train high-quality models quickly. Comparing DDLS side-by-side is difficult due to their extensive feature lists and architectural deviations. We aim to shine some light on the fundamental principles that are at work when training deep neural networks in a cluster of independent machines by analyzing the general properties associated with training deep learning models and how such workloads can be distributed in a cluster to achieve collaborative model training. Thereby we provide an overview of the different techniques that are used by contemporary DDLS and discuss their influence and implications on the training process. To conceptualize and compare DDLS, we group different techniques into categories, thus establishing a taxonomy of distributed deep learning systems .

Distributed Training of Deep Learning Models: A Taxonomic Perspective

Many distributed deep learning systems have been published over the past few years, often accompanied by impressive performance claims. In practice these figures are often achieved in high performance computing (HPC) environments with fast InfiniBand network connections. For average deep learning practitioners this is usually an unrealistic scenario, since they cannot afford access to these facilities. Simple re-implementations of algorithms such as EASGD  [1] for standard Ethernet environments often fail to replicate the scalability and performance of the original works [2] . In this paper, we explore this particular problem domain and present MPCA SGD, a method for distributed training of deep neural networks that is specifically designed to run in low-budget environments. MPCA SGD tries to make the best possible use of available resources, and can operate well if network bandwidth is constrained. Furthermore, MPCA SGD runs on top of the popular Apache Spark  [3] framework. Thus, it can easily be deployed in existing data centers and office environments where Spark is already used. When training large deep learning models in a gigabit Ethernet cluster, MPCA SGD achieves significantly faster convergence rates than many popular alternatives. For example, MPCA SGD can train ResNet-152  [4] up to 5.3x faster than state-of-the-art systems like MXNet  [5] , up to 5.3x faster than bulk-synchronous systems like SparkNet  [6] and up to 5.3x faster than decentral asynchronous systems like EASGD  [1] .

MPCA SGD—A Method for Distributed Training of Deep Learning Models on Spark

Distributed deep learning systems (DDLS) train deep neural network models by utilizing the distributed resources of a cluster Developers of DDLS are required to make many decisions to process their particular workloads in their chosen environment efficiently The advent of GPU-based deep learning, the ever-increasing size of datasets and deep neural network models, in combination with the bandwidth constraints that exist in cluster environments require developers of DDLS to be innovative in order to train high quality models quickly Comparing DDLS side-by-side is difficult due to their extensive feature lists and architectural deviations We aim to shine some light on the fundamental principles that are at work when training deep neural networks in a cluster of independent machines by analyzing the general properties associated with training deep learning models and how such workloads can be distributed in a cluster to achieve collaborative model training Thereby we provide an overview of the different techniques that are used by contemporary DDLS and discuss their influence and implications on the training process To conceptualize and compare DDLS, we group different techniques into categories, thus establishing a taxonomy of distributed deep learning systems

It is very important for swimming coaches to analyse a swimmer’s performance at the end of each race, since the analysis can then be used to change strategies for the next round. Coaches rely heavily on statistics, such as stroke length and instantaneous velocity, when analysing performance. These statistics are usually derived from time-consuming manual video annotations. To automatically obtain the required statistics from swimming videos, we need to solve the following four challenging computer vision tasks: swimmer head detection; tracking; stroke detection; and camera calibration. We collectively solve these problems using a two-phased deep learning approach, we call Deep Detector for Actions and Swimmer Heads (DeepDASH). DeepDASH achieves a 20.8% higher F1 score for swimmer head detection and operates 6 times faster than the popular Faster R-CNN object detector. We also propose a hierarchical tracking algorithm based on the existing SORT algorithm which we call HISORT. HISORT produces significantly longer tracks than SORT by preserving swimmer identities for longer periods of time. Finally, DeepDASH achieves an overall F1 score of 97.5% for stroke detection across all four swimming stroke styles.

Matthias Langer

Papers

Distributed Training of Deep Learning Models: A Taxonomic Perspective

MPCA SGD—A Method for Distributed Training of Deep Learning Models on Spark

Distributed Training of Deep Learning Models: A Taxonomic Perspective

The detection, tracking, and temporal action localisation of swimmers for automated analysis