Mean absolute percentage error

Abstract: The relative abilities of 2, dimensioned statistics — the root-mean-square error (RMSE) and the mean absolute error (MAE) — to describe average model-performance error are examined The RMSE is of special interest because it is widely reported in the climatic and environmental liter- ature; nevertheless, it is an inappropriate and misinterpreted measure of average error RMSE is inappropriate because it is a function of 3 characteristics of a set of errors, rather than of one (the average error) RMSE varies with the variability within the distribution of error magnitudes and with the square root of the number of errors (n 1/2 ), as well as with the average-error magnitude (MAE) Our findings indicate that MAE is a more natural measure of average error, and (unlike RMSE) is unambiguous Dimensioned evaluations and inter-comparisons of average model-performance error, therefore, should be based on MAE

Abstract: This study evaluated measures for making comparisons of errors across time series. We analyzed 90 annual and 101 quarterly economic time series. We judged error measures on reliability, construct validity, sensitivity to small changes, protection against outliers, and their relationship to decision making. The results lead us to recommend the Geometric Mean of the Relative Absolute Error (GMRAE) when the task involves calibrating a model for a set of time series. The GMRAE compares the absolute error of a given method to that from the random walk forecast. For selecting the most accurate methods, we recommend the Median RAE (MdRAE) when few series are available and the Median Absolute Percentage Error (MdAPE) otherwise. The Root Mean Square Error (RMSE) is not reliable, and is therefore inappropriate for comparing accuracy across series.

Abstract: Summary Developing a hydrological forecasting model based on past records is crucial to effective hydropower reservoir management and scheduling. Traditionally, time series analysis and modeling is used for building mathematical models to generate hydrologic records in hydrology and water resources. Artificial intelligence (AI), as a branch of computer science, is capable of analyzing long-series and large-scale hydrological data. In recent years, it is one of front issues to apply AI technology to the hydrological forecasting modeling. In this paper, autoregressive moving-average (ARMA) models, artificial neural networks (ANNs) approaches, adaptive neural-based fuzzy inference system (ANFIS) techniques, genetic programming (GP) models and support vector machine (SVM) method are examined using the long-term observations of monthly river flow discharges. The four quantitative standard statistical performance evaluation measures, the coefficient of correlation ( R ), Nash–Sutcliffe efficiency coefficient ( E ), root mean squared error (RMSE), mean absolute percentage error (MAPE), are employed to evaluate the performances of various models developed. Two case study river sites are also provided to illustrate their respective performances. The results indicate that the best performance can be obtained by ANFIS, GP and SVM, in terms of different evaluation criteria during the training and validation phases.

Abstract: Wind speed forecasting is critical for wind energy conversion systems since it greatly influences the issues such as the scheduling of a power system, and the dynamic control of the wind turbine In this paper, we present a comprehensive comparison study on the application of different artificial neural networks in 1-h-ahead wind speed forecasting Three types of typical neural networks, namely, adaptive linear element, back propagation, and radial basis function, are investigated The wind data used are the hourly mean wind speed collected at two observation sites in North Dakota The performance is evaluated based on three metrics, namely, mean absolute error, root mean square error, and mean absolute percentage error The results show that even for the same wind dataset, no single neural network model outperforms others universally in terms of all evaluation metrics Moreover, the selection of the type of neural networks for best performance is also dependent upon the data sources Among the optimal models obtained, the relative difference in terms of one particular evaluation metric can be as much as 20% This indicates the need of generating a single robust and reliable forecast by applying a post-processing method

Abstract: Short-term traffic flow prediction has long been regarded as a critical concern for intelligent transportation systems. On the basis of many existing prediction models, each having good performance only in a particular period, an improved approach is to combine these single predictors together for prediction in a span of periods. In this paper, a neural network model is introduced that combines the prediction from single neural network predictors according to an adaptive and heuristic credit assignment algorithm based on the theory of conditional probability and Bayes' rule. Two single predictors, i.e., the back propagation and the radial basis function neural networks are designed and combined linearly into a Bayesian combined neural network model. The credit value for each predictor in the combined model is calculated according to the proposed credit assignment algorithm and largely depends on the accumulative prediction perfor- mance of these predictors during the previous prediction intervals. For experimental test, two data sets comprising traffic flow rates in 15-min time intervals have been collected from Singapore's Ayer Rajah Expressway. One data set is used to train the two single neural networks and the other to test and compare the performances between the combined and singular models. Three indices, i.e., the mean absolute percentage error, the variance of absolute percentage error, and the probability of percentage error, are employed to compare the forecasting performance. It is found that most of the time, the combined model outperforms the singular predictors. More importantly, for a given time period, it is the role of this newly proposed model to track the predictors' performance online, so as to always select and combine the best-performing predictors for prediction.