Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Wireless sensor networks (WSNs) have been widely applied in various industrial applications, which involve collecting a massive amount of heterogeneous sensory data. However, most of the data-gathering strategies for WSNs cannot avoid the hotspot problem in local or whole deployment area. Hotspot problem affects the network connectivity and decreases the network lifetime. Hence, we propose a tree-cluster-based data-gathering algorithm (TCBDGA) for WSNs with a mobile sink. A novel weight-based tree-construction method is introduced. The root nodes of the constructed trees are defined as rendezvous points (RPs). Additionally, some special nodes called subrendezvous points (SRPs) are selected according to their traffic load and hops to root nodes. RPs and SRPs are viewed as stop points of the mobile sink for data collection, and can be reselected after a certain period. The simulation and comparison with other algorithms show that our TCBDGA can significantly balance the load of the whole network, reduce the energy consumption, alleviate the hotspot problem, and prolong the network lifetime.

A Tree-Cluster-Based Data-Gathering Algorithm for Industrial WSNs With a Mobile Sink

Dynamic clustering and management of mobile wireless sensor networks

Intelligent transportation (e.g., intelligent traffic light) makes our travel more convenient and efficient. With the development of mobile Internet and position technologies, it is reasonable to collect spatio-temporal data and then leverage these data to achieve the goal of intelligent transportation, and here, traffic prediction plays an important role. In this paper, we provide a comprehensive survey on traffic prediction, which is from the spatio-temporal data layer to the intelligent transportation application layer. At first, we split the whole research scope into four parts from bottom to up, where the four parts are, respectively, spatio-temporal data, preprocessing, traffic prediction and traffic application. Later, we review existing work on the four parts. First, we summarize traffic data into five types according to their difference on spatial and temporal dimensions. Second, we focus on four significant data preprocessing techniques: map-matching, data cleaning, data storage and data compression. Third, we focus on three kinds of traffic prediction problems (i.e., classification, generation and estimation/forecasting). In particular, we summarize the challenges and discuss how existing methods address these challenges. Fourth, we list five typical traffic applications. Lastly, we provide emerging research challenges and opportunities. We believe that the survey can help the partitioners to understand existing traffic prediction problems and methods, which can further encourage them to solve their intelligent transportation applications.

/pdf/a-survey-of-traffic-prediction-from-spatio-temporal-data-to-1n98645nzb.pdf

A Survey of Traffic Prediction: from Spatio-Temporal Data to Intelligent Transportation

Detecting anomalous traffic is a critical task for advanced Internet management. Many anomaly detection algorithms have been proposed recently. However, constrained by their matrix-based traffic data model, existing algorithms often suffer from low accuracy in anomaly detection. To fully utilize the multi-dimensional information hidden in the traffic data, this paper takes the initiative to investigate the potential and methodologies of performing tensor factorization for more accurate Internet anomaly detection. More specifically, we model the traffic data as a three-way tensor and formulate the anomaly detection problem as a robust tensor recovery problem with the constraints on the rank of the tensor and the cardinality of the anomaly set. These constraints, however, make the problem extremely hard to solve. Rather than resorting to the convex relaxation at the cost of low detection performance, we propose TensorDet to solve the problem directly and efficiently. To improve the anomaly detection accuracy and tensor factorization speed, TensorDet exploits the factorization structure with two novel techniques, sequential tensor truncation and two-phase anomaly detection. We have conducted extensive experiments using Internet traffic trace data Abilene and GEANT. Compared with the state of art algorithms for tensor recovery and matrix-based anomaly detection, TensorDet can achieve significantly lower false positive rate and higher true positive rate. Particularly, benefiting from our well designed algorithm to reduce the computation cost of tensor factorization, the tensor factorization process in TensorDet is 5 (Abilene) and 13 (GEANT) times faster than that of the traditional Tucker decomposition solution.

Fast Tensor Factorization for Accurate Internet Anomaly Detection

Network traffic data consists of Traffic Matrix (TM), which represents the volumes of traffic between Origin and Destination (OD) pairs in the network. It is a key input parameter of network engineering tasks. However, direct measurement of the OD pairs traffic is usually not feasible. Even good traffic measurement systems can suffer from errors, missing data. So obtaining the ODs traffic precisely is a challenge. Existing completion methods often perform poorly for network traffic estimation. Their recovery accuracy tends to be significantly worse when the data loss rate is high. Taking into account network traffic lower-dimensional latent structure and traffic hidden characteristic, a tensor (multi-way array) is introduced to model a time series of pure spatial traffic matrices in this paper. To recover the missing entries in tensors of traffic data, a novel spatio-temporal tensor completion method has been proposed. This approach not only takes advantage of tensor decomposition and its lower-dimensional representation, but also well takes into account traffic spatio-temporal properties. The extensive experiments with the real-world traffic trace data show that the proposed method can significantly reduce the missing traffic data recovery errors and achieve satisfactory completion accuracy comparing with the state-of-the-art completion methods.

Spatio-temporal tensor completion for imputing missing internet traffic data

Traffic matrix is an abstract representation of the traffic volume flowing between sets of source and destination pairs. It is a key input parameter of network operations management, planning, provisioning and traffic engineering. Traffic matrix is also important in the context of OpenFlow-based networks. Because even good measurement systems can suffer from errors and data collection systems can fail, missing values are common. Existing matrix completion methods do not consider traffic exhibit characteristics and only provide a finite precision. To address this problem, this paper proposes a novel approach based on compressive sensing and traffic self-similarity to reconstruct the missing traffic flow data. Firstly, we analyze the real-world traffic matrix, which all exhibit low-rank structure, temporal smoothness feature and spatial self-similarity. Then, we propose Self-Similarity and Temporal Compressive Sensing (SSTCS) algorithm to reconstruct the missing traffic data. The extensive experiments with the real-world traffic matrix show that our proposed SSTCS can significantly reduce data reconstruction errors and achieve satisfactory accuracy comparing with the existing solutions. Typically SSTCS can successfully reconstruct the traffic matrix with less than 32% errors when as much as 98% of the data is missing.

Data reconstruction in internet traffic matrix

Accurate traffic matrix completion based on multi-Gaussian models

The traffic volumes between a set of Origins and Destinations (OD) pairs within a network become increasingly critical for network operations management, planning, provisioning and traffic engineering. However, in practice it is challenging to reliably measure traffic in the whole network. For example, because of flaws in the measurement systems and attacks launched in a network, missing data and outliers are unavoidable. It is thus important to recover the missing entries and identify errors from the partial direct measurements. Existing recovery methods cannot sufficiently capture the multi-dimensional and spatial-temporal features of traffic data, which lead to perform poorly for network traffic data estimation. Their recovery accuracy tends to be significantly worse in the presence of both high data loss rate and gross corruptions. To address this problem, we propose a novel robust spatio-temporal tensor recovery (STTR) method to deal with missing data and outliers. First, by taking advantage of the structure in all dimensions of the traffic data, we organize the traffic data as a multi-way array (i.e., tensor). Second, by taking into account network traffic spatiotemporal characteristic, we incorporate domain knowledge about the structure of the underlying traffic data for missing values recovery and outlier removal. The proposed STTR is evaluated on real-world traffic trace data. Experimental results demonstrate that our STTR can achieve significantly better performance compared with the state-of-the-art recovery methods.

Huibin Zhou

Papers

Spatio-temporal tensor completion for imputing missing internet traffic data

Data reconstruction in internet traffic matrix

Accurate traffic matrix completion based on multi-Gaussian models

Robust Spatio-temporal Tensor Recovery for Internet Traffic Data