In this book, Andrew Harvey sets out to provide a unified and comprehensive theory of structural time series models. Unlike the traditional ARIMA models, structural time series models consist explicitly of unobserved components, such as trends and seasonals, which have a direct interpretation. As a result the model selection methodology associated with structural models is much closer to econometric methodology. The link with econometrics is made even closer by the natural way in which the models can be extended to include explanatory variables and to cope with multivariate time series. From the technical point of view, state space models and the Kalman filter play a key role in the statistical treatment of structural time series models. The book includes a detailed treatment of the Kalman filter. This technique was originally developed in control engineering, but is becoming increasingly important in fields such as economics and operations research. This book is concerned primarily with modelling economic and social time series, and with addressing the special problems which the treatment of such series poses. The properties of the models and the methodological techniques used to select them are illustrated with various applications. These range from the modellling of trends and cycles in US macroeconomic time series to to an evaluation of the effects of seat belt legislation in the UK.

Forecasting, Structural Time Series Models and the Kalman Filter

Highway Capacity Manual改訂の動向--テイラ-教授の講演より

527 articles related to imbalanced data and rare events are reviewed.Viewing reviewed papers from both technical and practical perspectives.Summarizing existing methods and corresponding statistics by a new taxonomy idea.Categorizing 162 application papers into 13 domains and giving introduction.Some opening questions are discussed at the end of this manuscript. Rare events, especially those that could potentially negatively impact society, often require humans decision-making responses. Detecting rare events can be viewed as a prediction task in data mining and machine learning communities. As these events are rarely observed in daily life, the prediction task suffers from a lack of balanced data. In this paper, we provide an in depth review of rare event detection from an imbalanced learning perspective. Five hundred and seventeen related papers that have been published in the past decade were collected for the study. The initial statistics suggested that rare events detection and imbalanced learning are concerned across a wide range of research areas from management science to engineering. We reviewed all collected papers from both a technical and a practical point of view. Modeling methods discussed include techniques such as data preprocessing, classification algorithms and model evaluation. For applications, we first provide a comprehensive taxonomy of the existing application domains of imbalanced learning, and then we detail the applications for each category. Finally, some suggestions from the reviewed papers are incorporated with our experiences and judgments to offer further research directions for the imbalanced learning and rare event detection fields.

Learning from class-imbalanced data

Accurate and real-time traffic forecasting plays an important role in the intelligent traffic system and is of great significance for urban traffic planning, traffic management, and traffic control. However, traffic forecasting has always been considered an “open” scientific issue, owing to the constraints of urban road network topological structure and the law of dynamic change with time. To capture the spatial and temporal dependences simultaneously, we propose a novel neural network-based traffic forecasting method, the temporal graph convolutional network (T-GCN) model, which is combined with the graph convolutional network (GCN) and the gated recurrent unit (GRU). Specifically, the GCN is used to learn complex topological structures for capturing spatial dependence and the gated recurrent unit is used to learn dynamic changes of traffic data for capturing temporal dependence. Then, the T-GCN model is employed to traffic forecasting based on the urban road network. Experiments demonstrate that our T-GCN model can obtain the spatio-temporal correlation from traffic data and the predictions outperform state-of-art baselines on real-world traffic datasets. Our tensorflow implementation of the T-GCN is available at https://www.github.com/lehaifeng/T-GCN .

T-GCN: A Temporal Graph Convolutional Network for Traffic Prediction

Spatiotemporal forecasting has various applications in neuroscience, climate and transportation domain. Traffic forecasting is one canonical example of such learning task. The task is challenging due to (1) complex spatial dependency on road networks, (2) non-linear temporal dynamics with changing road conditions and (3) inherent difficulty of long-term forecasting. To address these challenges, we propose to model the traffic flow as a diffusion process on a directed graph and introduce Diffusion Convolutional Recurrent Neural Network (DCRNN), a deep learning framework for traffic forecasting that incorporates both spatial and temporal dependency in the traffic flow. Specifically, DCRNN captures the spatial dependency using bidirectional random walks on the graph, and the temporal dependency using the encoder-decoder architecture with scheduled sampling. We evaluate the framework on two real-world large scale road network traffic datasets and observe consistent improvement of 12% - 15% over state-of-the-art baselines.

/pdf/diffusion-convolutional-recurrent-neural-network-data-driven-3wefkkjq0q.pdf

Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting

Impact of heterogeneity of car-following behavior on rear-end crash risk

Urban expressways usually experience several levels of service (LOS) because of the stop-and-go traffic flow caused by congestion. Moreover, multiple shock waves generate at different LOS interfaces. The dynamic of shock waves strongly influences the travel time reliability (TTR) of urban expressways. This study proposes a path TTR model that considers the dynamic of shock waves by using probability-based method to characterize the TTR of urban expressways with shock waves. Two model parameters are estimated, namely distribution of travel time (TT) per unit distance and travel distances in different LOS segments. Generalized extreme value distribution and generalized Pareto distribution are derived as distributions of TT per unit distance for six different LOS. Distribution parameters are estimated by using historical floating car data. Travel distances in different LOS segments are calculated based on shock wave theory. The range of TT along the path, which can help drivers arrange their trips, can be obtained from the TTR model. Finally, comparison is made among the proposed TTR model, generalized Pareto contrast model, which does not consider different LOS or existence of shock waves, and normal contrast model, which assumes TT per unit distance as normal distribution without considering shock wave. Results show that the proposed model achieves higher prediction accuracy and reduces the prediction range of TT. The conclusions can be further extended to TT prediction and assessment of measures to improve reliability of TT in a network.

A travel time reliability model of urban expressways with varying levels of service

The development of technologies related to connected and automated vehicles (CAVs) allows for a new approach to collect vehicle trajectory. However, trajectory data collected in this way represent only sampled traffic flow owing to low penetration rates (PRs) of CAVs and privacy concerns, and fail to provide a comprehensive picture of traffic flow. This study proposes a method to reconstruct vehicle trajectories in fully sampled traffic flow on freeways that consists of human-driven vehicles (HVs) and CAVs by using the mobile sensing data acquired from CAVs. The expected behavior of vehicles within the detection range of CAVs is determined based on the driving state classified by the Wiedemann model, i.e., free driving, emergency, closing, and following. If the actual behavior is different from the expected, it is deemed to be influenced by the undetected HVs. Then, new HVs are inserted based on the estimated local traffic density and speed of the freeway. The trajectories of the inserted HVs are further reconstructed by using the established update rules of cellular automation, i.e., uniform motion, acceleration, deceleration, randomization and position update. Last, the proposed method was examined by simulation experiments under different traffic densities and PRs of CAVs. The results show that the trajectories of fully sampled mixed traffic flow can be reconstructed reasonably well, not only under traffic conditions without explicit congestion but in congested environments.

Trajectory reconstruction for freeway traffic mixed with human-driven vehicles and connected and automated vehicles

The automatic number plate recognition (ANPR) system has been widely implemented as an important part of intelligent transportation system (ITS). However, similar to other traffic monitoring devices, missing data is a common and critical problem in the ANPR system. To solve the missing data problem, numerous tensor-based methods have been proposed in previous studies. Most of them, however, assume that where and when missing data occur in the dataset are known. This would be impractical, because missing data may occur randomly. In this study, we propose a novel tensor-based algorithm, specifically, an iterative tensor decomposition (ITD) approach, that utilizes multidimensional inherent correlation of traffic data to detect and impute missing data in the ANPR system. The proposed algorithm is tested with a real-world ANPR system dataset. The experimental results show that missing data from the ANPR system can be classified into three cases, i.e., no missing, random elements missing, and extreme missing. The proposed ITD can accurately detect and correct missing data under different missing cases. Furthermore, ITD is also compared with other state-of-the-art methods and the results show that ITD outperforms the existing methods.

Missing data detection and imputation for urban ANPR system using an iterative tensor decomposition approach

Many accidents occurring at signalized intersections are closely related to drivers' decisions of running through intersections during yellow light, i.e., yellow-light running (YLR). Therefore it is important to understand the relationships between YLR and the factors which contribute to drivers' decision of YLR. This requires collecting a large amount of YLR cases. However, existing data collection method, which mainly relies on video cameras, has difficulties to collect a large amount of YLR data. In this research, we propose a method to study drivers' YLR behaviors using high-resolution event-based data from signal control systems. We used 8 months' high-resolution data collected by two stop-bar detectors at a signalized intersection located in Minnesota and identified over 30,000 YLR cases. To identify the possible reasons for drivers' decision of YLR, this research further categorized the YLR cases into four types: "in should-go zone", "in should-stop zone", "in dilemma zone", and "in optional zone" according to the driver's location when signal turns to yellow. Statistical analysis indicates that the mean values of approaching speed and acceleration rate are significantly different for different types of YLR. We also show that there were about 10% of YLR drivers who cannot run through intersection before traffic light turns to red. Furthermore, based on a strong correlation between hourly traffic volume and number of YLR events, this research developed a regression model that can be used to predict the number of YLR events based on hourly flow rate. This research also showed that snowing weather conditions cause more YLR events.

Yunpeng Wang

Papers

Impact of heterogeneity of car-following behavior on rear-end crash risk

A travel time reliability model of urban expressways with varying levels of service

Trajectory reconstruction for freeway traffic mixed with human-driven vehicles and connected and automated vehicles

Missing data detection and imputation for urban ANPR system using an iterative tensor decomposition approach

Analysis of yellow-light running at signalized intersections using high-resolution traffic data