We present PyTorch Geometric Temporal a deep learning framework combining state-of-the-art machine learning algorithms for neural spatiotemporal signal processing. The main goal of the library is to make temporal geometric deep learning available for researchers and machine learning practitioners in a unified easy-to-use framework. PyTorch Geometric Temporal was created with foundations on existing libraries in the PyTorch eco-system, streamlined neural network layer definitions, temporal snapshot generators for batching, and integrated benchmark datasets. These features are illustrated with a tutorial-like case study. Experiments demonstrate the predictive performance of the models implemented in the library on real world problems such as epidemiological forecasting, ridehail demand prediction and web-traffic management. Our sensitivity analysis of runtime shows that the framework can potentially operate on web-scale datasets with rich temporal features and spatial structure.

PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models.

Due to great advances in wireless communication, the connected Internet of vehicles (CIoVs) has become prevalent. Naturally, internal connections among active vehicles are an indispensable factor in traffic forecasting. Although many related research studies have been conducted in the past few years, they mainly designed and/or developed single forecasting models for traffic forecasting. Such models may show ideal performance in some scenarios but lack satisfactory robustness to dynamic scenario changes. To address this challenge, a graph neural network-driven traffic forecasting model for CIoVs is proposed in this work, which is denoted as Gra-TF. In this paper, we regard the dynamics of traffic data as a temporal evolution scenario. With the assistance of ensemble learning, three typical graph-level prediction methods are employed to construct an integrated and enhanced forecasting model. This design utilizes several methods to minimize uncertainty in CIoVs. Finally, we use a real-world dataset to build an experimental scenario for further assessment. Numerical results indicate that the proposed Gra-TF improves the prediction accuracy by 30% to 40% compared with several baseline methods. 

Graph Neural Network-Driven Traffic Forecasting for the Connected Internet of Vehicles

Graph Convolutional Network (GCN) has been widely applied in transportation demand prediction due to its excellent ability to capture non-Euclidean spatial dependence among station-level or regional transportation demands. However, in most of the existing research, the graph convolution was implemented on a heuristically generated adjacency matrix, which could neither reflect the real spatial relationships of stations accurately, nor capture the multi-level spatial dependence of demands adaptively. To cope with the above problems, this paper provides a novel graph convolutional network for transportation demand prediction. Firstly, a novel graph convolution architecture is proposed, which has different adjacency matrices in different layers and all the adjacency matrices are self-learned during the training process. Secondly, a layer-wise coupling mechanism is provided, which associates the upper-level adjacency matrix with the lower-level one. It also reduces the scale of parameters in our model. Lastly, a unitary network is constructed to give the final prediction result by integrating the hidden spatial states with gated recurrent unit, which could capture the multi-level spatial dependence and temporal dynamics simultaneously. Experiments have been conducted on two real-world datasets, NYC Citi Bike and NYC Taxi, and the results demonstrate the superiority of our model over the state-of-the-art ones.

Coupled Layer-wise Graph Convolution for Transportation Demand Prediction

The Industrial Internet of Things (IIoT) plays an important role in digital transformation of traditional industries toward Industry 4.0. By connecting sensors, instruments, and other industry devices to the Internet, IIoT facilitates the data collection, data analysis, and automated control, thereby improving the productivity and efficiency of the business as well as the resulting economic benefits. Due to the complex IIoT infrastructure, anomaly detection becomes an important tool to ensure the success of IIoT. Due to the nature of IIoT, graph-level anomaly detection has been a promising means to detect and predict anomalies in many different domains, such as transportation, energy, and factory, as well as for dynamically evolving networks. This article provides a useful investigation on graph neural networks (GNNs) for anomaly detection in IIoT-enabled smart transportation, smart energy, and smart factory. In addition to the GNN-empowered anomaly detection solutions on point, contextual, and collective types of anomalies, useful data sets, challenges, and open issues for each type of anomalies in the three identified industry sectors (i.e., smart transportation, smart energy, and smart factory) are also provided and discussed, which will be useful for future research in this area. To demonstrate the use of GNN in concrete scenarios, we show three case studies in smart transportation, smart energy, and smart factory, respectively. 

/pdf/graph-neural-networks-for-anomaly-detection-in-industrial-2j409e5p.pdf

Graph Neural Networks for Anomaly Detection in Industrial Internet of Things

We present PyTorch Geometric Temporal, a deep learning framework combining state-of-the-art machine learning algorithms for neural spatiotemporal signal processing. The main goal of the library is to make temporal geometric deep learning available for researchers and machine learning practitioners in a unified easy-to-use framework. PyTorch Geometric Temporal was created with foundations on existing libraries in the PyTorch eco-system, streamlined neural network layer definitions, temporal snapshot generators for batching, and integrated benchmark datasets. These features are illustrated with a tutorial-like case study. Experiments demonstrate the predictive performance of the models implemented in the library on real-world problems such as epidemiological forecasting, ride-hail demand prediction, and web traffic management. Our sensitivity analysis of runtime shows that the framework can potentially operate on web-scale datasets with rich temporal features and spatial structure.

PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models

Deep spatio-temporal graph convolutional network for traffic accident prediction

Given a sequence of sets, where each set contains an arbitrary number of elements, the problem of temporal sets prediction aims to predict the elements in the subsequent set. In practice, temporal sets prediction is much more complex than predictive modelling of temporal events and time series, and is still an open problem. Many possible existing methods, if adapted for the problem of temporal sets prediction, usually follow a two-step strategy by first projecting temporal sets into latent representations and then learning a predictive model with the latent representations. The two-step approach often leads to information loss and unsatisfactory prediction performance. In this paper, we propose an integrated solution based on the deep neural networks for temporal sets prediction. A unique perspective of our approach is to learn element relationship by constructing set-level co-occurrence graph and then perform graph convolutions on the dynamic relationship graphs. Moreover, we design an attention-based module to adaptively learn the temporal dependency of elements and sets. Finally, we provide a gated updating mechanism to find the hidden shared patterns in different sequences and fuse both static and dynamic information to improve the prediction performance. Experiments on real-world data sets demonstrate that our approach can achieve competitive performances even with a portion of the training data and can outperform existing methods with a significant margin.

Predicting Temporal Sets with Deep Neural Networks

Accurate traffic forecasting is of vital importance for the management and decision in intelligent transportation systems. Indeed, it is a nontrivial endeavor to predict future traffic conditions due to the complexity of spatial relationships and temporal dependencies. Recent research developed Spatio-Temporal Graph Neural Networks (ST-GNNs) to capture the spatio-temporal correlations and achieved superior performance. However, the graph adjacency matrices that most ST-GNNs use are either pre-defined by heuristic rules or directly learned with trainable parameters. While node attributes, which record valuable information of traffic conditions, have not been fully exploited to guide the learning of better graph structure. In this paper, we propose an Adaptive Spatio-Temporal graph neural Network, namely Ada-STNet, to first derive optimal graph structure with the guidance of node attributes and then capture the complicated spatio-temporal correlations via a dedicated spatio-temporal convolution architecture for multi-step traffic condition forecasting. Specifically, we first propose a graph structure learning component to obtain an optimal graph adjacency matrix from both macro and micro perspectives. Next, we design a dedicated spatio-temporal convolution architecture to learn spatial relationships and temporal dependencies. Moreover, we present a two-stage training strategy to improve the model performance. Extensive experimental results on real-world datasets demonstrate the effectiveness and interpretability of our approach. 

Adaptive Spatio-temporal Graph Neural Network for traffic forecasting

So far COVID-19 has resulted in mass deaths and huge economic losses across the world. Various measures such as quarantine and social distancing have been taken to prevent the spread of this disease. These prevention measures have changed the transmission dynamics of COVID-19 and introduced new challenges for epidemic modelling and prediction. In this paper, we study a novel disease spreading model with two important aspects. First, the proposed model takes the quarantine effect of confirmed cases on transmission dynamics into account, which can better resemble the real-world scenario. Second, our model incorporates two types of human mobility, where the intra-region human mobility is related to the internal transmission speed of the disease in the focal area and the inter-region human mobility reflects the scale of external infectious sources to a focal area. With the proposed model, we use the human mobility data from 24 cities in China and 8 states in the USA to analyse the disease spreading patterns. The results show that our model could well fit/predict the reported cases in both countries. The predictions and findings shed light on how to effectively control COVID-19 by managing human mobility behaviours.

/pdf/modelling-the-epidemic-dynamics-of-covid-19-with-4lggxi3nwg.pdf

Le Yu

Papers

Deep spatio-temporal graph convolutional network for traffic accident prediction

Predicting Temporal Sets with Deep Neural Networks

Adaptive Spatio-temporal Graph Neural Network for traffic forecasting

Modelling the epidemic dynamics of COVID-19 with consideration of human mobility.

Predicting Temporal Sets with Deep Neural Networks