Data Mining - Concepts and Techniques.

Time series prediction techniques have been used in many real-world applications such as financial market prediction, electric utility load forecasting , weather and environmental state prediction, and reliability forecasting. The underlying system models and time series data generating processes are generally complex for these applications and the models for these systems are usually not known a priori. Accurate and unbiased estimation of the time series data produced by these systems cannot always be achieved using well known linear techniques, and thus the estimation process requires more advanced time series prediction algorithms. This paper provides a survey of time series prediction applications using a novel machine learning approach: support vector machines (SVM). The underlying motivation for using SVMs is the ability of this methodology to accurately forecast time series data when the underlying system processes are typically nonlinear, non-stationary and not defined a-priori. SVMs have also been proven to outperform other non-linear techniques including neural-network based non-linear prediction techniques such as multi-layer perceptrons.The ultimate goal is to provide the reader with insight into the applications using SVM for time series prediction, to give a brief tutorial on SVMs for time series prediction, to outline some of the advantages and challenges in using SVMs for time series prediction, and to provide a source for the reader to locate books, technical journals, and other online SVM research resources.

Time Series Prediction Using Support Vector Machines: A Survey

In almost every scientific field, measurements are performed over time. These observations lead to a collection of organized data called time series. The purpose of time-series data mining is to try to extract all meaningful knowledge from the shape of data. Even if humans have a natural capacity to perform these tasks, it remains a complex problem for computers. In this article we intend to provide a survey of the techniques applied for time-series data mining. The first part is devoted to an overview of the tasks that have captured most of the interest of researchers. Considering that in most cases, time-series task relies on the same components for implementation, we divide the literature depending on these common aspects, namely representation techniques, distance measures, and indexing methods. The study of the relevant literature has been categorized for each individual aspects. Four types of robustness could then be formalized and any kind of distance could then be classified. Finally, the study submits various research trends and avenues that can be explored in the near future. We hope that this article can provide a broad and deep understanding of the time-series data mining research field.

/pdf/time-series-data-mining-4v2qvzp3ln.pdf

Time-series data mining

Energy consumption forecasting for buildings has immense value in energy efficiency and sustainability research. Accurate energy forecasting models have numerous implications in planning and energy optimization of buildings and campuses. For new buildings, where past recorded data is unavailable, computer simulation methods are used for energy analysis and forecasting future scenarios. However, for existing buildings with historically recorded time series energy data, statistical and machine learning techniques have proved to be more accurate and quick. This study presents a comprehensive review of the existing machine learning techniques for forecasting time series energy consumption. Although the emphasis is given to a single time series data analysis, the review is not just limited to it since energy data is often co-analyzed with other time series variables like outdoor weather and indoor environmental conditions. The nine most popular forecasting techniques that are based on the machine learning platform are analyzed. An in-depth review and analysis of the ‘hybrid model’, that combines two or more forecasting techniques is also presented. The various combinations of the hybrid model are found to be the most effective in time series energy forecasting for building.

A review on time series forecasting techniques for building energy consumption

Multi-step ahead forecasting is still an open challenge in time series forecasting. Several approaches that deal with this complex problem have been proposed in the literature but an extensive comparison on a large number of tasks is still missing. This paper aims to fill this gap by reviewing existing strategies for multi-step ahead forecasting and comparing them in theoretical and practical terms. To attain such an objective, we performed a large scale comparison of these different strategies using a large experimental benchmark (namely the 111 series from the NN5 forecasting competition). In addition, we considered the effects of deseasonalization, input variable selection, and forecast combination on these strategies and on multi-step ahead forecasting at large. The following three findings appear to be consistently supported by the experimental results: Multiple-Output strategies are the best performing approaches, deseasonalization leads to uniformly improved forecast accuracy, and input selection is more effective when performed in conjunction with deseasonalization.

A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition

Methodology for long-term prediction of time series

This paper presents a comparison between direct and recursive prediction strategies. In order to perform the input selection, an approach based on mutual information is used. The mutual information is computed between all the possible input sets and the outputs. Least Squares Support Vector Machines are used as non-linear models to avoid local minima problems. Results are illustrated on the Poland electricity load benchmark and they show the superiority of the direct prediction strategy.

/pdf/direct-and-recursive-prediction-of-time-series-using-mutual-3jnshency8.pdf

Direct and recursive prediction of time series using mutual information selection

This paper presents a new method for the selection of the two hyperparameters of Least Squares Support Vector Machine (LS-SVM) approximators with Gaussian Kernels. The two hyperparameters are the width σ of the Gaussian kernels and the regularization parameter λ. For different values of σ, a Nonparametric Noise Estimator (NNE) is introduced to estimate the variance of the noise on the outputs. The NNE allows the determination of the best λ for each given σ. A Leave-one-out methodology is then applied to select the best σ. Therefore, this method transforms the double optimization problem into a single optimization one. The method is tested on 2 problems: a toy example and the Pumadyn regression Benchmark.

/pdf/ls-svm-hyperparameter-selection-with-a-nonparametric-noise-1i6b7yh3ru.pdf

LS-SVM hyperparameter selection with a nonparametric noise estimator

This paper presents k-NN as an approximator for time series prediction problems. The main advantage of this approximator is its simplicity. Despite the simplicity, k-NN can be used to perform input selection for nonlinear models and it also provides accurate approximations. Three model structure selection methods are presented: Leave-one-out, Bootstrap and Bootstrap 632. We will show that both Bootstraps provide a good estimate of the number of neighbors, k, where Leave-one-out fails. Results of the methods are presented with the Electric load from Poland data set.

/pdf/input-and-structure-selection-for-k-nn-approximator-zoxqiifxpp.pdf

Input and structure selection for k -NN approximator

We consider the stochastic matching problem. An edge-weighted general (i.e., not necessarily bipartite) graph G(V, E) is given in the input, where each edge in E is realized independently with probability p ; the realization is initially unknown, however, we are able to query the edges to determine whether they are realized. The goal is to query only a small number of edges to find a realized matching that is sufficiently close to the maximum matching among all realized edges. The stochastic matching problem has received a considerable attention during the past decade after the initial paper of Chen et al. [ICALP'09] because of its numerous real-world applications in kidney-exchange, matchmaking services, online labor markets, and advertisements. Most relevant to our work are the recent papers of Blum et al. [EC'15], Assadi et al. [EC'16, EC'17] and Maehara and Yamaguchi~[SODA'18] that consider the same model of stochastic matching. Our main result is an adaptive algorithm that for any arbitrarily small e > 0, finds a (1-e)-approximation in expectation, by querying only O(1) edges per vertex. We further show that our approach leads to a (1/2-e)-approximate non-adaptive algorithm that also uses $O(1)$ edges per vertex. Prior to our work, no nontrivial approximation was known for weighted graphs using a constant per-vertex budget. The state-of-the-art adaptive (resp. non-adaptive) algorithm of Maehara and Yamaguchi achieves a (1-e)-approximation (resp. (1/2-e)-approximation) by querying up to O(w logn) edges per vertex where w denotes the maximum integer edge-weight. Our result is a substantial improvement over this bound and has an appealing message: No matter what the structure of the input graph is, one can get arbitrarily close to the optimum solution by querying only a constant number of edges per vertex. To obtain our results, we introduce novel properties of a generalization of augmenting paths to weighted matchings that may be of independent interest.

Nima Reyhani

Papers

Methodology for long-term prediction of time series

Direct and recursive prediction of time series using mutual information selection

LS-SVM hyperparameter selection with a nonparametric noise estimator

Input and structure selection for k -NN approximator

Almost Optimal Stochastic Weighted Matching with Few Queries