The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time. Very few previous studies have examined this crucial and challenging weather forecasting problem from the machine learning perspective. In this paper, we formulate precipitation nowcasting as a spatiotemporal sequence forecasting problem in which both the input and the prediction target are spatiotemporal sequences. By extending the fully connected LSTM (FC-LSTM) to have convolutional structures in both the input-to-state and state-to-state transitions, we propose the convolutional LSTM (ConvLSTM) and use it to build an end-to-end trainable model for the precipitation nowcasting problem. Experiments show that our ConvLSTM network captures spatiotemporal correlations better and consistently outperforms FC-LSTM and the state-of-the-art operational ROVER algorithm for precipitation nowcasting.

Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

In Distributed Algorithms, Nancy Lynch provides a blueprint for designing, implementing, and analyzing distributed algorithms. She directs her book at a wide audience, including students, programmers, system designers, and researchers.



Distributed Algorithms contains the most significant algorithms and impossibility results in the area, all in a simple automata-theoretic setting. The algorithms are proved correct, and their complexity is analyzed according to precisely defined complexity measures. The problems covered include resource allocation, communication, consensus among distributed processes, data consistency, deadlock detection, leader election, global snapshots, and many others.



The material is organized according to the system model-first by the timing model and then by the interprocess communication mechanism. The material on system models is isolated in separate chapters for easy reference.



The presentation is completely rigorous, yet is intuitive enough for immediate comprehension. This book familiarizes readers with important problems, algorithms, and impossibility results in the area: readers can then recognize the problems when they arise in practice, apply the algorithms to solve them, and use the impossibility results to determine whether problems are unsolvable. The book also provides readers with the basic mathematical tools for designing new algorithms and proving new impossibility results. In addition, it teaches readers how to reason carefully about distributed algorithms-to model them formally, devise precise specifications for their required behavior, prove their correctness, and evaluate their performance with realistic measures.


Table of Contents

1 Introduction 
2 Modelling I; Synchronous Network Model 
3 Leader Election in a Synchronous Ring 
4 Algorithms in General Synchronous Networks 
5 Distributed Consensus with Link Failures 
6 Distributed Consensus with Process Failures 
7 More Consensus Problems 
8 Modelling II: Asynchronous System Model 
9 Modelling III: Asynchronous Shared Memory Model 
10 Mutual Exclusion 
11 Resource Allocation 
12 Consensus 
13 Atomic Objects 
14 Modelling IV: Asynchronous Network Model 
15 Basic Asynchronous Network Algorithms 
16 Synchronizers 
17 Shared Memory versus Networks 
18 Logical Time 
19 Global Snapshots and Stable Properties 
20 Network Resource Allocation 
21 Asynchronous Networks with Process Failures 
22 Data Link Protocols 
23 Partially Synchronous System Models 
24 Mutual Exclusion with Partial Synchrony 
25 Consensus with Partial Synchrony

Distributed algorithms

/pdf/convolutional-lstm-network-a-machine-learning-approach-for-3mrj9pedfq.pdf

Convolutional LSTM Network: a machine learning approach for precipitation nowcasting

Multiagent systems are made up of multiple interacting intelligent agents -- computational entities to some degree autonomous and able to cooperate, compete, communicate, act flexibly, and exercise control over their behavior within the frame of their objectives They are the enabling technology for a wide range of advanced applications relying on distributed and parallel processing of data, information, and knowledge relevant in domains ranging from industrial manufacturing to e-commerce to health care This book offers a state-of-the-art introduction to multiagent systems, covering the field in both breadth and depth, and treating both theory and practice It is suitable for classroom use or independent study This second edition has been completely revised, capturing the tremendous developments in multiagent systems since the first edition appeared in 1999 Sixteen of the book's seventeen chapters were written for this edition; all chapters are by leaders in the field, with each author contributing to the broad base of knowledge and experience on which the book rests The book covers basic concepts of computational agency from the perspective of both individual agents and agent organizations; communication among agents; coordination among agents; distributed cognition; development and engineering of multiagent systems; and background knowledge in logics and game theory Each chapter includes references, many illustrations and examples, and exercises of varying degrees of difficulty The chapters and the overall book are designed to be self-contained and understandable without additional material Supplemental resources are available on the book's Web site Contributors:Rafael Bordini, Felix Brandt, Amit Chopra, Vincent Conitzer, Virginia Dignum, Jurgen Dix, Ed Durfee, Edith Elkind, Ulle Endriss, Alessandro Farinelli, Shaheen Fatima, Michael Fisher, Nicholas R Jennings, Kevin Leyton-Brown, Evangelos Markakis, Lin Padgham, Julian Padget, Iyad Rahwan, Talal Rahwan, Alex Rogers, Jordi Sabater-Mir, Yoav Shoham, Munindar P Singh, Kagan Tumer, Karl Tuyls, Wiebe van der Hoek, Laurent Vercouter, Meritxell Vinyals, Michael Winikoff, Michael Wooldridge, Shlomo Zilberstein

Multiagent Systems

From the Publisher:
Among the many approaches to formal reasoning about programs, Dynamic Logic enjoys the singular advantage of being strongly related to classical logic. Its variants constitute natural generalizations and extensions of classical formalisms. For example, Propositional Dynamic Logic (PDL) can be described as a blend of three complementary classical ingredients: propositional calculus, modal logic, and the algebra of regular events. In First-Order Dynamic Logic (DL), the propositional calculus is replaced by classical first-order predicate calculus. Dynamic Logic is a system of remarkable unity that is theoretically rich as well as of practical value. It can be used for formalizing correctness specifications and proving rigorously that those specifications are met by a particular program. Other uses include determining the equivalence of programs, comparing the expressive power of various programming constructs, and synthesizing programs from specifications. 
This book provides the first comprehensive introduction to Dynamic Logic. It is divided into three parts. The first part reviews the appropriate fundamental concepts of logic and computability theory and can stand alone as an introduction to these topics. The second part discusses PDL and its variants, and the third part discusses DL and its variants. Examples are provided throughout, and exercises and a short historical section are included at the end of each chapter.

Dynamic Logic

We suggest a new simple forwarding technique to speed up IP destination address lookup. The technique is a natural extension of IP, requires 5 bits in the IP header (IPv4, 7 in IPv6), and performs IP lookup nearly as fast as IP/Tag switching but with a smaller memory requirement and a much simpler protocol. The basic idea is that each router adds a "clue" to each packet, telling its downstream router where it ended the IP lookup. Since the forwarding tables of neighboring routers are similar, the clue either directly determines the best prefix match for the downstream router, or provides the downstream router with a good point to start its IP lookup. The new scheme thus prevents repeated computations and distributes the lookup process across the routers along the packet path. Each router starts the lookup computation at the point its upstream neighbor has finished. Furthermore, the new scheme is easily assimilated into heterogeneous IP networks, does not require routers coordination, and requires no setup time. Even a flow of one packet enjoys the benefits of the scheme without any additional overhead. The speedup we achieve is about 10 times faster than current standard techniques. In a sense, this paper shows that the current routers employed in the Internet are clue-less; namely, it is possible to speed up the IP lookup by an order of magnitude without any major changes to the existing protocols.

Routing with a clue

We present view transactions, a model for relaxed consistency checks in software transactional memory (STM). View transactions always operate on a consistent snapshot of memory but may commit in a different snapshot. They are therefore simpler to reason about, provide opacity and maintain composability. In addition, view transactions avoid many of the overheads associated with previous approaches for relaxing consistency checks. As a result, view transactions outperform the prior approaches by 1.13x to 2x on various benchmarks.

/pdf/brief-announcement-view-transactions-transactional-model-4dkjmq96nj.pdf

Brief announcement: view transactions: transactional model with relaxed consistency checks

This paper introduces a new network model called a multimedia network. It combines the point-to-point message passing network and the multiaccess channel. To benefit from the combination, the designed algorithms consist of two stages: a local stage which utilizes the parallelism of the point-to-point network and a global stage which utilizes the broadcast capability of the multiaccess channel. To balance the complexities of the two stages a partition of the network into O( n ) connected components each of radius O( n ) is required. We present efficient deterministic and randomized partitioning algorithms that run in ( n log ∗ n ) time. The deterministic algorithm sends O(m + n log n log ∗ n) messages, while the randomized algorithm sends only O(m + n log ∗ n) messages. (n and m are the number of nodes and point-to-point links in the network.) The partitioning algorithms are then used to obtain: (1) O( n log n log ∗ n ) time deterministic and O( n log ∗ n ) time randomized algorithms for computing global sensitive functions, and (2) An O( n log n ) time deterministic algorithm for computing a minimum spanning tree. We give Ω(n) time lower bounds for computing global sensitive functions in both point-to-point and multiaccess networks, thus showing that the multimedia network is more powerful than both its separate components. Furthermore, we prove and Ω( n ) time lower bound for computing global sensitive functions in multimedia networks, thus leaving a small gap between our upper and lower bounds.

The power of multimedia: combining point-to-point and multiaccess networks

We present a basic tool for zero day attack signature extraction. Given two large sets of messages, $P$ the messages captured in the network at peacetime (i.e., mostly legitimate traffic) and $A$ the messages captured during attack time (i.e., contains many attack messages), we present a tool for extracting a set $S$ of strings that are frequently found in $A$ and not in $P$ , thus allowing the identification of the attack packets. This is an important tool in protecting sites on the Internet from worm attacks and distributed denial of service attacks and may also be useful for other problems, including command and control identification and the DNA-sequences analysis. The main contributions of this paper are the system we developed to extract the required signatures together with the string-heavy hitters problem definition and the algorithm for solving this problem. This algorithm finds popular strings of variable length in a set of messages, using, in a tricky way, the classic heavy-hitter algorithm as a building block. The algorithm runs in linear time requiring one-pass over the input. Our system makes use of this algorithm to extract the desired signatures. Furthermore, we provide an extended algorithm which is able to identify groups of signatures, often found together in the same packets, which further improves the quality of signatures generated by our system. Using our system, a yet unknown attack can be detected and stopped within minutes from attack start time.

Zero-Day Signature Extraction for High-Volume Attacks

We present a new system to protect IoT devices in multiple premises by a single Virtual Network Function (VNF) deployed in the ISP network. The system is based on the Manufacturer Usage Description (MUD) framework, a white-list IoT protection scheme that has been proposed in recent years.While MUD is designed for on-premise deployment, here we adapt it to work as a scalable, managed service in the ISP level. Our service does not require any cooperation or installation on the client premise or on the IoT devices themselves. Furthermore, it monitors the IoT traffic and detects malicious behavior, including outgoing DDoS traffic, without being on the critical path, and it filters bad traffic by ACLs on either the POP router or the client CPE. The CPE itself is considered an IoT device and traffic destined or that originates at the CPE is monitored as well. For the white-list method we extend the MUD architectural framework to support peer to peer communicating IoT devices (e.g., direct mobile device to IoT device communication).The system includes a mechanism to distinguish between flows of different devices at the ISP level despite the fact that most home networks (and their IoT devices) are behind a NAT and all the flows from the same home come out with the same source IP address. Moreover, the NFV system needs to receive only the first packet of each flow/connection at the VNF, and rules space is proportional to the number of unique types of IoT devices rather than the total number of IoT devices (which is much larger).A PoC with a large national level ISP proves that our technology works as expected, identifying the various IoT devices that are connected to the network and detecting any unauthorized communications.

Yehuda Afek

Papers

Routing with a clue

Brief announcement: view transactions: transactional model with relaxed consistency checks

The power of multimedia: combining point-to-point and multiaccess networks

Zero-Day Signature Extraction for High-Volume Attacks

NFV-based IoT Security for Home Networks using MUD