Showing papers on "Asynchronous communication published in 2019"

Using Asynchronous Audio Feedback to Enhance Teaching Presence and Students' Sense of Community.

Abstract: This paper reports the findings of a case study in which audio feedback replaced text-based feedback in asynchronous courses. Previous research has demonstrated that participants in online courses can build effective learning communities through text-based communication alone. Similarly, it has been demonstrated that instructors for online courses can adequately project immediacy behaviors using text-based communication. However, we believed that the inclusion of an auditory element might strengthen both the sense of community and the instructor’s ability to affect more personalized communication with students. Over the course of one semester, students in this study received a mixture of asynchronous audio and text-based feedback. Our findings revealed extremely high student satisfaction with embedded asynchronous audio feedback as compared to asynchronous text only feedback. Four themes, which accounted for this preference, were culled out in an iterative, inductive analysis of interview data: 1. Audio feedback was perceived to be more effective than text-based feedback for conveying nuance; 2. Audio feedback was associated with feelings of increased involvement and enhanced learning community interactions; 3. Audio feedback was associated with increased retention of content; and 4. Audio feedback was associated with the perception that the instructor cared more about the student. Document analysis revealed that students were three times more likely to apply content for which audio commenting was provided in class projects than was the case for content for which text-based commenting was provided. Audio commenting was also found to significantly increase the level at which students applied such content. Implications of this case study and directions for future research are addressed in the discussion and conclusions section of this paper.

Asynchronous federated optimization

Abstract: Federated learning enables training on a massive number of edge devices. To improve flexibility and scalability, we propose a new asynchronous federated optimization algorithm. We prove that the proposed approach has near-linear convergence to a global optimum, for both strongly convex and a restricted family of non-convex problems. Empirical results show that the proposed algorithm converges quickly and tolerates staleness in various applications.

Asynchronous Finite-Time Filtering of Networked Switched Systems and its Application: an Event-Driven Method

Abstract: To effectively use the network resources, a suitable event-driven communication scheme is proposed for the networked switched systems in this paper. Under the EDCS, a finite-time filter is designed for switched systems, which does not synchronize with the switched systems. Different from the existing finite-time problems, finite-time boundedness (FTBs) and input–output finite-time stability (IO-FTSy) are simultaneously considered in this paper. Some sufficient conditions are established to check the properties of the FTBs and the IO-FTSy of the event-driven asynchronous filtering error system by constructing a reasonable Lyapunov–Krasovskii functional and using the average dwell time approach. All the matrix inequalities can be converted to linear matrix inequalities so as to design the event-driven asynchronous filter. The applicability of the proposed filtering scheme is demonstrated via a mass-spring system model.

End-to-End Learning of Representations for Asynchronous Event-Based Data

Abstract: Event cameras are vision sensors that record asynchronous streams of per-pixel brightness changes, referred to as "events”. They have appealing advantages over frame based cameras for computer vision, including high temporal resolution, high dynamic range, and no motion blur. Due to the sparse, non-uniform spatio-temporal layout of the event signal, pattern recognition algorithms typically aggregate events into a grid-based representation and subsequently process it by a standard vision pipeline, e.g., Convolutional Neural Network (CNN). In this work, we introduce a general framework to convert event streams into grid-based representations by means of strictly differentiable operations. Our framework comes with two main advantages: (i) allows learning the input event representation together with the task dedicated network in an end to end manner, and (ii) lays out a taxonomy that unifies the majority of extant event representations in the literature and identifies novel ones. Empirically, we show that our approach to learning the event representation end-to-end yields an improvement of approximately 12% on optical flow estimation and object recognition over state-of-the-art methods.

Sampled-Data Adaptive Output Feedback Fuzzy Stabilization for Switched Nonlinear Systems With Asynchronous Switching

Abstract: This paper considers the problem of sampled-data adaptive output feedback fuzzy stabilization for switched uncertain nonlinear systems associated with asynchronous switching. A state observer is designed to estimate the unmeasured states and fuzzy logic systems are employed to deal with the unknown nonlinear terms. Sampled-data controller and novel switched adaptive laws are constructed based on the recursive design method and an average dwell time constraint is given to ensure that the closed-loop system is stable. The proposed scheme is employed in a mass–spring–damper system to demonstrate its effectiveness.

Event-Triggered-Based Distributed Cooperative Energy Management for Multienergy Systems

Abstract: This paper investigates the issues of day-ahead and real-time cooperative energy management for multienergy systems formed by many energy bodies. To address these issues, we propose an event-triggered-based distributed algorithm with some desirable features, namely, distributed execution, asynchronous communication, and independent calculation. First, the energy body, seen as both energy supplier and customer, is introduced for system model development. On this basis, energy bodies cooperate with each other to achieve the objective of maximizing the day-ahead social welfare and smoothing out the real-time loads variations as well as renewable resource fluctuations with the consideration of different timescale characteristics between electricity and heat power. To this end, the day-ahead and real-time energy management models are established and formulated as a class of distributed coupled optimization problem by felicitously converting some system coordinates. Such problems can be effectively solved by implementing the proposed algorithm. With the effort, each energy body can determine its owing optimal operations through only local communication and computation, resulting in enhanced system reliability, scalability, and privacy. Meanwhile, the designed communication strategy is event-triggered, which can dramatically reduce the communication among energy bodies. Simulations are provided to illustrate the effectiveness of the proposed models and algorithm.

Event/Self-Triggered Control for Leader-Following Consensus Over Unreliable Network With DoS Attacks

Abstract: This paper investigates the leader-following consensus issue with event/self-triggered schemes under an unreliable network environment. First, we characterize network communication and control protocol update in the presence of denial-of-service (DoS) attacks. In this situation, an event-triggered communication scheme is first proposed to effectively schedule information transmission over the network possibly subject to malicious attacks. In this communication framework, synchronous and asynchronous updated strategies of control protocols are constructed to achieve leader-following consensus in the presence of DoS attacks. Moreover, to further reduce the cost induced by event detection, a self-triggered communication scheme is proposed in which the next triggering instant can be determined by computing with the most updated information. Finally, a numerical example is provided to verify the effectiveness of the proposed communication schemes and updated strategies in the unreliable network environment.

FDML: A Collaborative Machine Learning Framework for Distributed Features

Abstract: Most current distributed machine learning systems try to scale up model training by using a data-parallel architecture that divides the computation for different samples among workers. We study distributed machine learning from a different motivation, where the information about the same samples, e.g., users and objects, are owned by several parities that wish to collaborate but do not want to share raw data with each other. We propose an asynchronous stochastic gradient descent (SGD) algorithm for such a feature distributed machine learning (FDML) problem, to jointly learn from distributed features, with theoretical convergence guarantees under bounded asynchrony. Our algorithm does not require sharing the original features or even local model parameters between parties, thus preserving the data locality. The system can also easily incorporate differential privacy mechanisms to preserve a higher level of privacy. We implement the FDML system in a parameter server architecture and compare our system with fully centralized learning (which violates data locality) and learning based on only local features, through extensive experiments performed on both a public data set a9a, and a large dataset of 5,000,000 records and 8700 decentralized features from three collaborating apps at Tencent including Tencent MyApp, Tecent QQ Browser and Tencent Mobile Safeguard. Experimental results have demonstrated that the proposed FDML system can be used to significantly enhance app recommendation in Tencent MyApp by leveraging user and item features from other apps, while preserving the locality and privacy of features in each individual app to a high degree.

Asymptotically Optimal Validated Asynchronous Byzantine Agreement

Abstract: We provide a new protocol for Validated Asynchronous Byzantine Agreement in the authenticated setting. Validated (multi-valued) Asynchronous Byzantine Agreement is a key building block in constructing Atomic Broadcast and fault-tolerant state machine replication in the asynchronous setting. Our protocol has optimal resilience of ƒ

GraphQ: Scalable PIM-Based Graph Processing

Abstract: Processing-In-Memory (PIM) architectures based on recent technology advances (e.g., Hybrid Memory Cube) demonstrate great potential for graph processing. However, existing solutions did not address the key challenge of graph processing---irregular data movements. This paper proposes GraphQ, an improved PIM-based graph processing architecture over recent architecture Tesseract, that fundamentally eliminates irregular data movements. GraphQ is inspired by ideas from distributed graph processing and irregular applications to enable static and structured communication with runtime and architecture co-design. Specifically, GraphQ realizes: 1) batched and overlapped inter-cube communication by reordering vertex processing order; 2) streamlined inter-cube communication by using heterogeneous cores for different access types. Moreover, to tackle the discrepancy between inter-cube and inter-node bandwidth, we propose a hybrid execution model that performs additional local computation during the inter-node communication. This model is general enough and applicable to asynchronous iterative algorithms that can tolerate bounded stale values. Putting all together, GraphQ simultaneously maximizes intra-cube, inter-cube, and inter-node communication throughput. In a zSim-based simulator with five real-world graphs and four algorithms, GraphQ achieves on average 3.3× and maximum 13.9× speedup, 81% energy saving compared with Tesseract. We show that increasing memory size in PIM also proportionally increases compute capability: a 4-node GraphQ achieves 98.34× speedup compared with a single node with the same memory size and conventional memory hierarchy.

Asynchronous Online Federated Learning for Edge Devices with Non-IID Data.

Abstract: Federated learning (FL) is a machine learning paradigm where a shared central model is learned across distributed edge devices while the training data remains on these devices. Federated Averaging (FedAvg) is the leading optimization method for training non-convex models in this setting with a synchronized protocol. However, the assumptions made by FedAvg are not realistic given the heterogeneity of devices. In particular, the volume and distribution of collected data vary in the training process due to different sampling rates of edge devices. The edge devices themselves also vary in their available communication bandwidth and system configurations, such as memory, processor speed, and power requirements. This leads to vastly different training times as well as model/data transfer times. Furthermore, availability issues at edge devices can lead to a lack of contribution from specific edge devices to the federated model. In this paper, we present an Asynchronous Online Federated Learning (ASO-Fed) framework, where the edge devices perform online learning with continuous streaming local data and a central server aggregates model parameters from clients. Our framework updates the central model in an asynchronous manner to tackle the challenges associated with both varying computational loads at heterogeneous edge devices and edge devices that lag behind or dropout. We perform extensive experiments on a simulated benchmark image dataset and three real-world non-IID streaming datasets. The results demonstrate the effectiveness of \model~on converging fast and maintaining good prediction performance.

A Bi-layered Parallel Training Architecture for Large-Scale Convolutional Neural Networks

Abstract: Benefitting from large-scale training datasets and the complex training network, Convolutional Neural Networks (CNNs) are widely applied in various fields with high accuracy. However, the training process of CNNs is very time-consuming, where large amounts of training samples and iterative operations are required to obtain high-quality weight parameters. In this paper, we focus on the time-consuming training process of large-scale CNNs and propose a Bi-layered Parallel Training (BPT-CNN) architecture in distributed computing environments. BPT-CNN consists of two main components: (a) an outer-layer parallel training for multiple CNN subnetworks on separate data subsets, and (b) an inner-layer parallel training for each subnetwork. In the outer-layer parallelism, we address critical issues of distributed and parallel computing, including data communication, synchronization, and workload balance. A heterogeneous-aware Incremental Data Partitioning and Allocation (IDPA) strategy is proposed, where large-scale training datasets are partitioned and allocated to the computing nodes in batches according to their computing power. To minimize the synchronization waiting during the global weight update process, an Asynchronous Global Weight Update (AGWU) strategy is proposed. In the inner-layer parallelism, we further accelerate the training process for each CNN subnetwork on each computer, where computation steps of convolutional layer and the local weight training are parallelized based on task-parallelism. We introduce task decomposition and scheduling strategies with the objectives of thread-level load balancing and minimum waiting time for critical paths. Extensive experimental results indicate that the proposed BPT-CNN effectively improves the training performance of CNNs while maintaining the accuracy.

Accelerating distributed reinforcement learning with in-switch computing

Abstract: Reinforcement learning (RL) has attracted much attention recently, as new and emerging AI-based applications are demanding the capabilities to intelligently react to environment changes. Unlike distributed deep neural network (DNN) training, the distributed RL training has its unique workload characteristics - it generates orders of magnitude more iterations with much smaller sized but more frequent gradient aggregations. More specifically, our study with typical RL algorithms shows that their distributed training is latency critical and that the network communication for gradient aggregation occupies up to 83.2% of the execution time of each training iteration. In this paper, we present iSwitch, an in-switch acceleration solution that moves the gradient aggregation from server nodes into the network switches, thus we can reduce the number of network hops for gradient aggregation. This not only reduces the end-to-end network latency for synchronous training, but also improves the convergence with faster weight updates for asynchronous training. Upon the in-switch accelerator, we further reduce the synchronization overhead by conducting on-the-fly gradient aggregation at the granularity of network packets rather than gradient vectors. Moreover, we rethink the distributed RL training algorithms and also propose a hierarchical aggregation mechanism to further increase the parallelism and scalability of the distributed RL training at rack scale. We implement iSwitch using a real-world programmable switch NetFPGA board. We extend the control and data plane of the programmable switch to support iSwitch without affecting its regular network functions. Compared with state-of-the-art distributed training approaches, iSwitch offers a system-level speedup of up to 3.66× for synchronous distributed training and 3.71× for asynchronous distributed training, while achieving better scalability.

Dual-Stream Recurrent Neural Network for Video Captioning

Abstract: Recent progress in using recurrent neural networks (RNNs) for video description has attracted an increasing interest, due to its capability to encode a sequence of frames for caption generation While existing methods have studied various features (eg, CNN, 3D CNN, and semantic attributes) for visual encoding, the representation and fusion of heterogeneous information from multi-modal spaces have not fully explored Consider that different modalities are often asynchronous, frame-level multi-modal fusion (eg, concatenation and linear fusion) will negatively influence each modality In this paper, we propose a dual-stream RNN (DS-RNN) framework to jointly discover and integrate the hidden states of both visual and semantic streams for video caption generation First, an encoding RNN is used for each stream to flexibly exploit the hidden states of respective modality Specifically, we proposed an attentive multi-grained encoder module to enhance the local feature learning with global semantics feature Then, a dual-stream decoder is deployed to integrate the asynchronous yet complementary sequential hidden states from both streams for caption generation Extensive experiments on three benchmark datasets, namely, MSVD, MSR-VTT, and MPII-MD, show that DS-RNN achieves competitive performance against the state-of-the-art Additional ablation studies were conducted on various variants of the proposed DS-RNN

Distributed Machine Learning with a Serverless Architecture

Abstract: The need to scale up machine learning, in the presence of a rapid growth of data both in volume and in variety, has sparked broad interests to develop distributed machine learning systems, typically based on parameter servers. However, since these systems are based on a dedicated cluster of physical or virtual machines, they have posed non-trivial cluster management overhead to machine learning practitioners and data scientists. In addition, there exists an inherent mismatch between the dynamically varying resource demands during a model training job and the inflexible resource provisioning model of current cluster-based systems.In this paper, we propose SIREN, an asynchronous distributed machine learning framework based on the emerging serverless architecture, with which stateless functions can be executed in the cloud without the complexity of building and maintaining virtual machine infrastructures. With SIREN, we are able to achieve a higher level of parallelism and elasticity by using a swarm of stateless functions, each working on a different batch of data, while greatly reducing system configuration overhead. Furthermore, we propose a scheduler based on Deep Reinforcement Learning to dynamically control the number and memory size of the stateless functions that should be used in each training epoch. The scheduler learns from the training process itself, in pursuit for the minimum possible training time given a cost. With our real-world prototype implementation on AWS Lambda, extensive experimental results have shown that SIREN can reduce model training time by up to 44%, as compared to traditional machine learning training benchmarks on AWS EC2 at the same cost.

Asynchronous Convolutional Networks for Object Detection in Neuromorphic Cameras

Abstract: Event-based cameras, also known as neuromorphic cameras, are bioinspired sensors able to perceive changes in the scene at high frequency with low power consumption. Becoming available only very recently, a limited amount of work addresses object detection on these devices. In this paper we propose two neural networks architectures for object detection: YOLE, which integrates the events into surfaces and uses a frame-based model to process them, and fcYOLE, an asynchronous event-based fully convolutional network which uses a novel and general formalization of the convolutional and max pooling layers to exploit the sparsity of camera events. We evaluate the algorithm with different extensions of publicly available datasets, and on a novel synthetic dataset.

A Low-Latency Communication Scheme for Mobile Wireless Sensor Control Systems

Abstract: Millions of dedicated sensors are deployed in smart cities to enhance quality of urban living. Communication technologies are critical for connecting these sensors and transmitting events to sink. In control systems of mobile wireless sensor networks (MWSNs), mobile nodes are constantly moving to detect events, while static nodes constitute the communication infrastructure for information transmission. Therefore, how to communicate with sink quickly and effectively is an important research issue for control systems of MWSNs. In this paper, a communication scheme named first relay node selection based on fast response and multihop relay transmission with variable duty cycle (FRAVD) is proposed. The scheme can effectively reduce the network delay by combining first relay node selection with node duty cycles setting. In FRAVD scheme, first, for the first relay node selection, we propose a strategy based on fast response, that is, select the first relay node from adjacent nodes in the communication range within the shortest response time, and guarantee that the remaining energy and the distance from sink of the node are better than the average. Then for multihop data transmission of static nodes, variable duty cycle is introduced novelty, which utilizes the residual energy to improve the duty cycle of nodes in far-sink area, because nodes adopt a sleep-wake asynchronous mode, increasing the duty cycle can significantly improve network performance in terms of delays and transmission reliability. Our comprehensive performance analysis has demonstrated that compared with the communication scheme with fixed duty cycle, the FRAVD scheme reduces the network delay by 24.17%, improves the probability of finding first relay node by 17.68%, while also ensuring the network lifetime is not less than the previous researches, and is a relatively efficient low-latency communication scheme.

Coordinated Tracking Control With Asynchronous Edge-Based Event-Triggered Communications

Abstract: This paper studies the distributed event-triggered tracking control problem of general linear multiagent systems with a dynamic leader, whose control input might be nonzero and unknown. The existence of the leader's unknown input renders the network heterogeneous and largely increases the difficulty of designing distributed event-based protocols. To solve this problem, we establish a novel asynchronous edge-based event-triggered mechanism, under which communication is not required from the leader to its out-neighbors (informed followers) or by the edge, unless the embedded triggering functions are triggered. Using this mechanism, we design a static asynchronous edge-based event-triggered protocol and an adaptive one, both of which guarantee the achievement of the leader–follower consensus and the exclusion of the Zeno behavior. Note that the adaptive event-based protocol is fully distributed, requiring no global information of the network topology, such as the network's scale, the smallest nonzero eigenvalue of the Laplacian matrix, and the upper bound of the leader's control input.

Hidden-Markov-Model-Based Asynchronous Filter Design of Nonlinear Markov Jump Systems in Continuous-Time Domain

Abstract: This paper addresses the dissipative asynchronous filtering problem for a class of Takagi–Sugeno fuzzy Markov jump systems in the continuous-time domain. The hidden Markov model is applied to describe the asynchronous situation between the designed filter and the original system. Based on the stochastic Lyapunov function, a sufficient condition is developed to guarantee the stochastic stability of the filtering error systems with a given dissipative performance. Two different methods for the existence of desired filter are established. Due to the Finsler’s lemma, the second approach has fewer variables to decide and brings less conservatism than the first one. Finally, an example is provided to demonstrate the correctness and advantage of the proposed approaches.

Finite-Time Resilient Controller Design of a Class of Uncertain Nonlinear Systems With Time-Delays Under Asynchronous Switching

Abstract: This paper investigates the asynchronous resilient controller design problem for a class of nonlinear switched systems with time-delays and uncertainties in a given finite-time interval. By constructing proper multiple Lyapunov–Krasovskii functions and applying average dwell time methods, a switching law and the relevant asynchronous resilient controller are designed to guarantee the finite-time boundedness of the closed-loop system with a specified ${H} _{\infty }$ performance index. The ${H} _{\infty }$ resilient controller design problems can be derived by solving a set of linear matrix inequalities. A practical example is employed to demonstrate the availability of the proposed methods.

SmartCC: A Reinforcement Learning Approach for Multipath TCP Congestion Control in Heterogeneous Networks

Abstract: The Multipath TCP (MPTCP) protocol has been standardized by the IETF as an extension of conventional TCP, which enables multi-homed devices to establish multiple paths for simultaneous data transmission. Congestion control is a fundamental mechanism for the design and implementation of MPTCP. Due to the diverse QoS characteristics of heterogeneous links, existing multipath congestion control mechanisms suffer from a number of performance problems such as bufferbloat, suboptimal bandwidth usage, etc. In this paper, we propose a learning-based multipath congestion control approach called SmartCC to deal with the diversities of multiple communication path in heterogeneous networks. SmartCC adopts an asynchronous reinforcement learning framework to learn a set of congestion rules, which allows the sender to observe the environment and take actions to adjust the subflows’ congestion windows adaptively to fit different network situations. To deal with the problem of infinite states in high-dimensional space, we propose a hierarchical tile coding algorithm for state aggregation and a function estimation approach for $Q$ -learning, which can derive the optimal policy efficiently. Due to the asynchronous design of SmartCC, the processes of model training and execution are decoupled, and the learning process will not introduce extra delay and overhead on the decision making process in MPTCP congestion control. We conduct extensive experiments for performance evaluation, which show that SmartCC improves the aggregate throughput significantly and outperforms the state-of-the-art mechanisms on a variety of performance metrics.

BugsJS: a Benchmark of JavaScript Bugs

Abstract: JavaScript is a popular programming language that is also error-prone due to its asynchronous, dynamic, and loosely-typed nature. In recent years, numerous techniques have been proposed for analyzing and testing JavaScript applications. However, our survey of the literature in this area revealed that the proposed techniques are often evaluated on different datasets of programs and bugs. The lack of a commonly used benchmark limits the ability to perform fair and unbiased comparisons for assessing the efficacy of new techniques. To fill this gap, we propose BugsJS, a benchmark of 453 real, manually validated JavaScript bugs from 10 popular JavaScript server-side programs, comprising 444k LOC in total. Each bug is accompanied by its bug report, the test cases that detect it, as well as the patch that fixes it. BugsJS features a rich interface for accessing the faulty and fixed versions of the programs and executing the corresponding test cases, which facilitates conducting highly-reproducible empirical studies and comparisons of JavaScript analysis and testing tools.

SparCML: high-performance sparse communication for machine learning

Abstract: Applying machine learning techniques to the quickly growing data in science and industry requires highly-scalable algorithms. Large datasets are most commonly processed "data parallel" distributed across many nodes. Each node's contribution to the overall gradient is summed using a global allreduce. This allreduce is the single communication and thus scalability bottleneck for most machine learning workloads. We observe that frequently, many gradient values are (close to) zero, leading to sparse of sparsifyable communications. To exploit this insight, we analyze, design, and implement a set of communication-efficient protocols for sparse input data, in conjunction with efficient machine learning algorithms which can leverage these primitives. Our communication protocols generalize standard collective operations, by allowing processes to contribute arbitrary sparse input data vectors. Our generic communication library, SparCML1, extends MPI to support additional features, such as non-blocking (asynchronous) operations and low-precision data representations. As such, SparCML and its techniques will form the basis of future highly-scalable machine learning frameworks.

Asynchronous Resilient Output Consensus of Switched Heterogeneous Linear Multivehicle Systems With Communication Delay

Abstract: The resilient output consensus problem for a network of switched heterogeneous linear vehicle systems with asynchronous switching, controller gain variation, unknown disturbance, and communication delay is investigated. Based on the topology decoupling technique, the investigated problem is transformed into two subproblems: one is a robust asynchronous static output feedback (SOF) control problem and the other is an asynchronous SOF problem. Based on the piecewise Lyapunov functional method and the average dwell time switching scheme, some new sufficient conditions are proposed for the solutions to those subproblems. Moreover, some novel linear-matrix-inequality-based conditions are developed to calculate the SOF controller gains. A network of six vehicles is finally presented to show the effectiveness of the proposed resilient consensus controller.

A Novel Asynchronous Control for Artificial Delayed Markovian Jump Systems via Output Feedback Sliding Mode Approach

Abstract: A novel asynchronous control for a class of Markovian jump systems (MJSs) via output feedback sliding mode approach with an artificial time-delay is proposed. The asynchronous control strategy is adopted owing to the nonsynchronization between the controlled system and the controller. In some practical applications, the state variables are often difficult to be measured directly from the outside of the system, which makes the implementation of state feedback technique more complex. However, the output information is always accessible to the system. Therefore, an asynchronous output feedback sliding controller, where an artificial time-delay is introduced in the synthesis of the sliding surface, for MJSs is designed to guarantee the sliding mode dynamics satisfying the reaching condition, and a sufficient condition is derived to ensure the resultant system exponentially stable. Besides, a program of optimization is given to optimize the artificial delay-time. Finally, a numerical simulation and a practical application are given to validate the effectiveness of the proposed technique.