THE DESIGN AND ANALYSIS OF EXPERIMENTS. By Oscar Kempthorne. New York, John Wiley and Sons, Inc., 1952. 631 pp. $8.50. This book by a teacher of statistics (as well as a consultant for \"experimenters\") is a comprehensive study of the philosophical background for the statistical design of experiment. It is necessary to have some facility with algebraic notation and manipulation to be able to use the volume intelligently. The problems are presented from the theoretical point of view, without such practical examples as would be helpful for those not acquainted with mathematics. The mathematical justification for the techniques is given. As a somewhat advanced treatment of the design and analysis of experiments, this volume will be interesting and helpful for many who approach statistics theoretically as well as practically. With emphasis on the \"why,\" and with description given broadly, the author relates the subject matter to the general theory of statistics and to the general problem of experimental inference. MARGARET J. ROBERTSON

The Design and Analysis of Experiments

Due to the simplicity of their implementations, least square support vector machine (LS-SVM) and proximal support vector machine (PSVM) have been widely used in binary classification applications. The conventional LS-SVM and PSVM cannot be used in regression and multiclass classification applications directly, although variants of LS-SVM and PSVM have been proposed to handle such cases. This paper shows that both LS-SVM and PSVM can be simplified further and a unified learning framework of LS-SVM, PSVM, and other regularization algorithms referred to extreme learning machine (ELM) can be built. ELM works for the “generalized” single-hidden-layer feedforward networks (SLFNs), but the hidden layer (or called feature mapping) in ELM need not be tuned. Such SLFNs include but are not limited to SVM, polynomial network, and the conventional feedforward neural networks. This paper shows the following: 1) ELM provides a unified learning platform with a widespread type of feature mappings and can be applied in regression and multiclass classification applications directly; 2) from the optimization method point of view, ELM has milder optimization constraints compared to LS-SVM and PSVM; 3) in theory, compared to ELM, LS-SVM and PSVM achieve suboptimal solutions and require higher computational complexity; and 4) in theory, ELM can approximate any target continuous function and classify any disjoint regions. As verified by the simulation results, ELM tends to have better scalability and achieve similar (for regression and binary class cases) or much better (for multiclass cases) generalization performance at much faster learning speed (up to thousands times) than traditional SVM and LS-SVM.

Extreme Learning Machine for Regression and Multiclass Classification

In this paper a novel nature-inspired optimization paradigm is proposed called Moth-Flame Optimization (MFO) algorithm. The main inspiration of this optimizer is the navigation method of moths in nature called transverse orientation. Moths fly in night by maintaining a fixed angle with respect to the moon, a very effective mechanism for travelling in a straight line for long distances. However, these fancy insects are trapped in a useless/deadly spiral path around artificial lights. This paper mathematically models this behaviour to perform optimization. The MFO algorithm is compared with other well-known nature-inspired algorithms on 29 benchmark and 7 real engineering problems. The statistical results on the benchmark functions show that this algorithm is able to provide very promising and competitive results. Additionally, the results of the real problems demonstrate the merits of this algorithm in solving challenging problems with constrained and unknown search spaces. The paper also considers the application of the proposed algorithm in the field of marine propeller design to further investigate its effectiveness in practice. Note that the source codes of the MFO algorithm are publicly available at http://www.alimirjalili.com/MFO.html.

Moth-flame optimization algorithm

Survey: Reservoir computing approaches to recurrent neural network training

Computational intelligence techniques have been used in wide applications. Out of numerous computational intelligence techniques, neural networks and support vector machines (SVMs) have been playing the dominant roles. However, it is known that both neural networks and SVMs face some challenging issues such as: (1) slow learning speed, (2) trivial human intervene, and/or (3) poor computational scalability. Extreme learning machine (ELM) as emergent technology which overcomes some challenges faced by other techniques has recently attracted the attention from more and more researchers. ELM works for generalized single-hidden layer feedforward networks (SLFNs). The essence of ELM is that the hidden layer of SLFNs need not be tuned. Compared with those traditional computational intelligence techniques, ELM provides better generalization performance at a much faster learning speed and with least human intervene. This paper gives a survey on ELM and its variants, especially on (1) batch learning mode of ELM, (2) fully complex ELM, (3) online sequential ELM, (4) incremental ELM, and (5) ensemble of ELM.

Extreme learning machines: a survey

In this paper, a robust recurrent neural network is presented in a Bayesian framework based on echo state mechanisms. Since the new model is capable of handling outliers in the training data set, it is termed as a robust echo state network (RESN). The RESN inherits the basic idea of ESN learning in a Bayesian framework, but replaces the commonly used Gaussian distribution with a Laplace one, which is more robust to outliers, as the likelihood function of the model output. Moreover, the training of the RESN is facilitated by employing a bound optimization algorithm, based on which, a proper surrogate function is derived and the Laplace likelihood function is approximated by a Gaussian one, while remaining robust to outliers. It leads to an efficient method for estimating model parameters, which can be solved by using a Bayesian evidence procedure in a fully autonomous way. Experimental results show that the proposed method is robust in the presence of outliers and is superior to existing methods.

Chaotic Time Series Prediction Based on a Novel Robust Echo State Network

A novel chaotic time-series prediction method based on support vector machines (SVMs) and echo-state mechanisms is proposed. The basic idea is replacing "kernel trick" with "reservoir trick" in dealing with nonlinearity, that is, performing linear support vector regression (SVR) in the high-dimension "reservoir" state space, and the solution benefits from the advantages from structural risk minimization principle, and we call it support vector echo-state machines (SVESMs). SVESMs belong to a special kind of recurrent neural networks (RNNs) with convex objective function, and their solution is global, optimal, and unique. SVESMs are especially efficient in dealing with real life nonlinear time series, and its generalization ability and robustness are obtained by regularization operator and robust loss function. The method is tested on the benchmark prediction problem of Mackey-Glass time series and applied to some real life time series such as monthly sunspots time series and runoff time series of the Yellow River, and the prediction results are promising

/pdf/support-vector-echo-state-machine-for-chaotic-time-series-1ym41jbxhr.pdf

Support Vector Echo-State Machine for Chaotic Time-Series Prediction

Chaos limits predictability so that the long-term prediction of chaotic time series is very difficult. The main purpose of this paper is to study a new methodology to model and predict chaotic time series based on a new recurrent predictor neural network (RPNN). This method realizes long-term prediction by making accurate multistep predictions. This RPNN consists of nonlinearly operated nodes whose outputs are only connected with the inputs of themselves and the latter nodes. The connections may contain multiple branches with time delays. An extended algorithm of self-adaptive back-propagation through time (BPTT) learning algorithm is used to train the RPNN. In simulation, two performance measures [root-mean-square error (RMSE) and prediction accuracy (PA)] show that the proposed method is more effective and accurate for multistep prediction. It can identify the systems characteristics quite well and provide a new way to make long-term prediction of the chaotic time series.

Prediction of chaotic time series based on the recurrent predictor neural network

In this paper, a cooperative time-varying formation maneuvering problem with connectivity preservation and collision avoidance is investigated for a fleet of autonomous surface vehicles (ASVs) with position–heading measurements. Each vehicle is subject to unknown kinetics induced by internal model uncertainty and external disturbances. At first, a nonlinear state observer is used to recover the unmeasured linear velocity and yaw rate as well as unknown uncertainty and disturbances. Then, observer-based cooperative time-varying formation maneuvering control laws are designed based on artificial potential functions, nonlinear tracking differentiators, and a backstepping technique. The stability of closed-loop distributed formation control system is analyzed based on input-to-state stability and cascade stability. The salient features of the proposed method are as follows. First, cooperative time-varying formation maneuvering with the capability of connectivity preservation and collision avoidance can be achieved in the absence of velocity measurements. Second, the complexity of the cooperative time-varying formation maneuvering control laws is reduced without resorting to dynamic surface control. Third, the uncertainty and disturbance are actively rejected in the presence of position–heading measurements. Simulation results are given to substantiate the proposed output feedback control method for cooperative time-varying formation maneuvering of ASVs with connectivity preservation and collision avoidance.

Min Han

Papers

Chaotic Time Series Prediction Based on a Novel Robust Echo State Network

Support Vector Echo-State Machine for Chaotic Time-Series Prediction

Prediction of chaotic time series based on the recurrent predictor neural network

Output-Feedback Cooperative Formation Maneuvering of Autonomous Surface Vehicles With Connectivity Preservation and Collision Avoidance

Online sequential extreme learning machine with kernels for nonstationary time series prediction