Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data to build feature-based classifiers and nonlinear predictive models. Training neural networks involves the optimization of nonconvex objective functions, and usually, the learning process is costly and infeasible for applications associated with data streams. A possible, albeit counterintuitive, alternative is to randomly assign a subset of the networks' weights so that the resulting optimization task can be formulated as a linear least-squares problem. This methodology can be applied to both feedforward and recurrent networks, and similar techniques can be used to approximate kernel functions. Many experimental results indicate that such randomized models can reach sound performance compared to fully adaptable ones, with a number of favorable benefits, including 1 simplicity of implementation, 2 faster learning with less intervention from human beings, and 3 possibility of leveraging overall linear regression and classification algorithms e.g., i¾?1 norm minimization for obtaining sparse formulations. This class of neural networks attractive and valuable to the data mining community, particularly for handling large scale data mining in real-time. However, the literature in the field is extremely vast and fragmented, with many results being reintroduced multiple times under different names. This overview aims to provide a self-contained, uniform introduction to the different ways in which randomization can be applied to the design of neural networks and kernel functions. A clear exposition of the basic framework underlying all these approaches helps to clarify innovative lines of research, open problems, and most importantly, foster the exchanges of well-known results throughout different communities. WIREs Data Mining Knowl Discov 2017, 7:e1200. doi: 10.1002/widm.1200

Randomness in neural networks: an overview

The key component in forecasting demand and consumption of resources in a supply network is an accurate prediction of real-valued time series. Indeed, both service interruptions and resource waste can be reduced with the implementation of an effective forecasting system. Significant research has thus been devoted to the design and development of methodologies for short term load forecasting over the past decades. A class of mathematical models, called Recurrent Neural Networks, are nowadays gaining renewed interest among researchers and they are replacing many practical implementation of the forecasting systems, previously based on static methods. Despite the undeniable expressive power of these architectures, their recurrent nature complicates their understanding and poses challenges in the training procedures. Recently, new important families of recurrent architectures have emerged and their applicability in the context of load forecasting has not been investigated completely yet. In this paper we perform a comparative study on the problem of Short-Term Load Forecast, by using different classes of state-of-the-art Recurrent Neural Networks. We test the reviewed models first on controlled synthetic tasks and then on different real datasets, covering important practical cases of study. We provide a general overview of the most important architectures and we define guidelines for configuring the recurrent networks to predict real-valued time series.

An overview and comparative analysis of Recurrent Neural Networks for Short Term Load Forecasting

Online sequential extreme learning machine with kernels for nonstationary time series prediction

When chaotic systems are used in different practical applications, such as nonlinear control and cryptography, their complex chaos dynamics are strongly required. However, many existing chaotic systems have simple complexity, and this brings negative effects to chaos-based applications. To address this issue, this paper introduces a sine chaotification model (SCM) as a general framework to enhance the chaos complexity of existing one-dimensional (1-D) chaotic maps. The SCM uses a sine function as a nonlinear chaotification transform and applies it to the output of a 1-D chaotic map. The resulting enhanced chaotic map of the SCM has better chaos complexity and a much larger chaotic range than the seed map. Theoretical analysis verifies the efficiency of the SCM. To show the performance of the SCM, we apply SCM to three existing chaotic maps and analyze the dynamics properties of the obtained enhanced chaotic maps. Performance evaluations prove that the three enhanced chaotic maps have more complicated dynamics behaviors than their seed chaotic maps. To show the implementation simplicity of the SCM, we implement the three enhanced chaotic maps using the field-programmable gate array. To investigate the SCM in practical application, we design pseudorandom number generators using the enhanced chaotic maps.

Sine Chaotification Model for Enhancing Chaos and Its Hardware Implementation

In order to approximate the underlying process of temporal data, time series prediction has been a hot research topic for decades. Developing predictive models plays an important role in interpreting complex real-world elements. With the sharp increase in the quantity and dimensionality of data, new challenges, such as extracting deep features and recognizing deep latent patterns, have emerged, demanding novel approaches and effective solutions. Deep learning, composed of multiple processing layers to learn with multiple levels of abstraction, is, now, commonly deployed for overcoming the newly arisen difficulties. This paper reviews the state-of-the-art developments in deep learning for time series prediction. Based on modeling for the perspective of conditional or joint probability, we categorize them into discriminative, generative, and hybrids models. Experiments are implemented on both benchmarks and real-world data to elaborate the performance of the representative deep learning-based prediction methods. Finally, we conclude with comments on possible future perspectives and ongoing challenges with time series prediction.

A Review of Deep Learning Models for Time Series Prediction

In this paper, a robust recurrent neural network is presented in a Bayesian framework based on echo state mechanisms. Since the new model is capable of handling outliers in the training data set, it is termed as a robust echo state network (RESN). The RESN inherits the basic idea of ESN learning in a Bayesian framework, but replaces the commonly used Gaussian distribution with a Laplace one, which is more robust to outliers, as the likelihood function of the model output. Moreover, the training of the RESN is facilitated by employing a bound optimization algorithm, based on which, a proper surrogate function is derived and the Laplace likelihood function is approximated by a Gaussian one, while remaining robust to outliers. It leads to an efficient method for estimating model parameters, which can be solved by using a Bayesian evidence procedure in a fully autonomous way. Experimental results show that the proposed method is robust in the presence of outliers and is superior to existing methods.

Chaotic Time Series Prediction Based on a Novel Robust Echo State Network

Abstract It is a difficult task to root the cause of the failure of a gas face seal because different causes may result in similar observations. In the work being presented, the discrimination of multiple types of abnormities in a spiral groove gas face seal is studied. A dynamic model is employed to analyze groups of cases in order to uncover the dynamic behaviors when the face contact is induced by different mixtures of abnormities, whose discriminative features when motion and contact are monitored are studied and uncovered. A circumferential-pattern-related oscillation phenomenon is discovered, which is extracted from contact information and implies the relative magnitude of the moment on stator and the rotor tilt. The experimental observation shows consistent results. It means that the grooves (or other circumferential patterns) generate useful informative features for monitoring. These results provide guidance for designing a monitored gas face seal system. 

/pdf/discriminative-features-of-abnormities-in-a-spiral-groove-21zrzcmx.pdf

Discriminative Features of Abnormities in a Spiral Groove Gas Face Seal Based on Dynamic Model Considering Contact

The path planning for mobile robots has attracted extensive attention, and evolutionary algorithms have been applied to this problem increasingly. In this paper, we propose a novel gradient eigendecomposition invariance biogeography-based optimization (GEI-BBO) for mobile robot path planning, which has the merits of high rotation invariance and excellent search performance. In GEI-BBO, we design an eigendecomposition mechanism for migration operation, which can reduce the dependency of biogeography-based optimization (BBO) on the coordinate system, improve the rotation invariance and share the information between eigensolutions more effectively. Meanwhile, to find the local optimal solution better, gradient descent is added, and the system search strategy can reduce the occurrence of local trapping phenomenon. In addition, combining the GEI-BBO with cubic spline interpolation will solve the problem of mobile robot path planning through a defined coding method and fitness function. A series of experiments are implemented on benchmark functions, whose results indicated that the optimization performance of GEI-BBO is superior to other algorithms. And the successful application of GEI-BBO for path planning in different environments confirms its effectiveness and practicability. 

/pdf/gradient-eigendecomposition-invariance-biogeography-based-2dbl95lk.pdf

Gradient eigendecomposition invariance biogeography-based optimization for mobile robot path planning

/pdf/discriminative-features-of-abnormities-in-a-spiral-groove-gjoskxy8.pdf

Mobile robot path planning (MRPP) plays an irreplaceable role in the process of intelligent robots and practical artificial intelligence. The traditional global path planning methods have some shortcomings, such as difficulty in digging into environmental information and finding the optimal path effectively. To solve the above problems, this paper proposes a negative gradient differential biogeography-based optimization (NG-DBBO), which has strong local search ability and global optimization ability. Firstly, we present a differential migration approach to increase the population diversity in the iterative process of NG-DBBO, which can realize the information sharing between feature solutions effectively. Then a negative gradient descent strategy based on negative gradient descent is introduced to improve the learning rate, which not only enhances initial global search ability, but also avoids premature convergence. Noteworthily, the convergence of the algorithm is analyzed for single-peak and multi-peak problems respectively. After that, NG-DBBO is combined with the cubic spline interpolation to realize MRPP by the defined coding method and fitness function. The simulation experiments are used to demonstrate the availability of our method, which consist of two parts. In the first part, we select 23 benchmark functions to verify the accuracy and convergence speed of the NG-DBBO algorithm. The practicability of path planning in different environments is demonstrated in the second part.

Decai Li

Papers

Chaotic Time Series Prediction Based on a Novel Robust Echo State Network

Discriminative Features of Abnormities in a Spiral Groove Gas Face Seal Based on Dynamic Model Considering Contact

Gradient eigendecomposition invariance biogeography-based optimization for mobile robot path planning

Discriminative Features of Abnormities in a Spiral Groove Gas Face Seal Based on Dynamic Model Considering Contact

Negative Gradient Differential Biogeography-based Optimization for Mobile Robot Path Planning