Introduction to support vector learning roadmap. Part 1 Theory: three remarks on the support vector method of function estimation, Vladimir Vapnik generalization performance of support vector machines and other pattern classifiers, Peter Bartlett and John Shawe-Taylor Bayesian voting schemes and large margin classifiers, Nello Cristianini and John Shawe-Taylor support vector machines, reproducing kernel Hilbert spaces, and randomized GACV, Grace Wahba geometry and invariance in kernel based methods, Christopher J.C. Burges on the annealed VC entropy for margin classifiers - a statistical mechanics study, Manfred Opper entropy numbers, operators and support vector kernels, Robert C. Williamson et al. Part 2 Implementations: solving the quadratic programming problem arising in support vector classification, Linda Kaufman making large-scale support vector machine learning practical, Thorsten Joachims fast training of support vector machines using sequential minimal optimization, John C. Platt. Part 3 Applications: support vector machines for dynamic reconstruction of a chaotic system, Davide Mattera and Simon Haykin using support vector machines for time series prediction, Klaus-Robert Muller et al pairwise classification and support vector machines, Ulrich Kressel. Part 4 Extensions of the algorithm: reducing the run-time complexity in support vector machines, Edgar E. Osuna and Federico Girosi support vector regression with ANOVA decomposition kernels, Mark O. Stitson et al support vector density estimation, Jason Weston et al combining support vector and mathematical programming methods for classification, Bernhard Scholkopf et al.

Advances in kernel methods: support vector learning

Chaotic time series data are observed routinely in experiments on physical systems and in observations in the field. The authors review developments in the extraction of information of physical importance from such measurements. They discuss methods for (1) separating the signal of physical interest from contamination ("noise reduction"), (2) constructing an appropriate state space or phase space for the data in which the full structure of the strange attractor associated with the chaotic observations is unfolded, (3) evaluating invariant properties of the dynamics such as dimensions, Lyapunov exponents, and topological characteristics, and (4) model making, local and global, for prediction and other goals. They briefly touch on the effects of linearly filtering data before analyzing it as a chaotic time series. Controlling chaotic physical systems and using them to synchronize and possibly communicate between source and receiver is considered. Finally, chaos in space-time systems, that is, the dynamics of fields, is briefly considered. While much is now known about the analysis of observed temporal chaos, spatio-temporal chaotic systems pose new challenges. The emphasis throughout the review is on the tools one now has for the realistic study of measured data in laboratory and field settings. It is the goal of this review to bring these tools into general use among physicists who study classical and semiclassical systems. Much of the progress in studying chaotic systems has rested on computational tools with some underlying rigorous mathematics. Heuristic and intuitive analysis tools guided by this mathematics and realizable on existing computers constitute the core of this review.

The analysis of observed chaotic data in physical systems

The European Centre for Medium-Range Weather Forecasts (ECMWF) Ensemble Prediction System (EPS) is described. In addition to an unperturbed (control) forecast, each ensemble comprises 32 10-day forecasts starting from initial conditions in which dynamically defined perturbations have been added to the operational analysis. The perturbations are constructed from singular vectors of a time-evolution operator linearized around the short-range-forecast trajectory. These singular vectors approximately determine the most unstable phase-space directions in the early part of the forecast period, and are estimated using a forward and adjoint linear version of the ECMWF numerical weather-prediction model. An appropriate norm is chosen, and relationships between the structures of these singular vectors at initial time and patterns showing the sensitivity of short-range forecast error to changes in the analysis are discussed. A methodology to perform a phase-space rotation of the singular vectors is described, which generates hemispheric-wide perturbations and renormalizes them according to analysis-error estimates from the data-assimilation system.

The validation of the ensembles is given firstly in terms of scatter diagrams and contingency tables of ensemble spread and control-forecast skill. The contingency tables are compared with those from a perfect-model ensemble system; no significant differences are found in some cases. Brier scores for the probability of European flow clusters are presented, which indicate predictive skill up to forecast-day 8 with respect to climatological probabilities. The dependence of these scores on flow-dependent model errors is also discussed. Finally, ensemble-member skill-score distributions are presented, which confirm the overall satisfactory performance of the EPS, particularly in summer and autumn 1993. In winter, cases of poor performance over Europe were associated with the occurrence of a split westerly flow with a blocking high and/or a cut-off low in the verifying analysis.

Two cases are studied in detail, one having large ensemble dispersion, the other corresponding to a more predictable situation. The case studies are used to illustrate the range of ensemble products routinely disseminated to ECMWF Member States. These products include clusters of flow types, and probability fields of weather elements.

The ECMWF Ensemble Prediction System: Methodology and validation

Surrogate time series

: This thesis applies neural network feature selection techniques to multivariate time series data to improve prediction of a target time series. Two approaches to feature selection are used. First, a subset enumeration method is used to determine which financial indicators are most useful for aiding in prediction of the S&P 500 futures daily price. The candidate indicators evaluated include RSI, Stochastics and several moving averages. Results indicate that the Stochastics and RSI indicators result in better prediction results than the moving averages. The second approach to feature selection is calculation of individual saliency metrics. A new decision boundary-based individual saliency metric, and a classifier independent saliency metric are developed and tested. Ruck's saliency metric, the decision boundary based saliency metric, and the classifier independent saliency metric are compared for a data set consisting of the RSI and Stochastics indicators as well as delayed closing price values. The decision based metric and the Ruck metric results are similar, but the classifier independent metric agrees with neither of the other metrics. The nine most salient features, determined by the decision boundary based metric, are used to train a neural network and the results are presented and compared to other published results. (AN)

/pdf/nonlinear-time-series-analysis-1j3a35n4y3.pdf

Nonlinear Time Series Analysis.

We examine the issue of determining an acceptable minimum embedding dimension by looking at the behavior of near neighbors under changes in the embedding dimension from d\ensuremath{\rightarrow}d+1. When the number of nearest neighbors arising through projection is zero in dimension ${\mathit{d}}_{\mathit{E}}$, the attractor has been unfolded in this dimension. The precise determination of ${\mathit{d}}_{\mathit{E}}$ is clouded by ``noise,'' and we examine the manner in which noise changes the determination of ${\mathit{d}}_{\mathit{E}}$. Our criterion also indicates the error one makes by choosing an embedding dimension smaller than ${\mathit{d}}_{\mathit{E}}$. This knowledge may be useful in the practical analysis of observed time series.

Determining embedding dimension for phase-space reconstruction using a geometrical construction

The time delay reconstruction of the state space of a system from observed scalar data requires a time lag and an integer embedding dimension. The minimum necessary global embedding dimension d E may still be larger than the actual dimension of the underlying dynamics d L . The embedding theorem only guarantees that the attractor of the system is fully unfolded using d E greater than 2d A , with d A the fractal attractor dimension. Using the idea of local false nearest neighbors, we discuss methods for determining the integer-valued d L

Local false nearest neighbors and dynamical dimensions from observed chaotic data

We propose a low-dimensional, physically motivated, nonlinear map as a model for cyclic combustion variation in spark-ignited internal combustion engines. A key feature is the interaction between stochastic, small-scale fluctuations in engine parameters and nonlinear deterministic coupling between successive engine cycles. Residual cylinder gas from each cycle alters the in-cylinder fuel-air ratio and thus the combustion efficiency in succeeding cycles. The model’s simplicity allows rapid simulation of thousands of engine cycles, permitting statistical studies of cyclic-variation patterns and providing physical insight into this technologically important phenomenon. Using symbol statistics to characterize the noisy dynamics, we find good quantitative matches between our model and experimental time-series measurements. @S1063-651X~98!08903-X#

Observing and modeling nonlinear dynamics in an internal combustion engine

The time delay reconstruction of the state space of a system from observed scalar data requires a time lag and an integer embedding dimension. The minimum necessary global embedding dimension ${\mathit{d}}_{\mathit{E}}$ may still be larger than the actual dimension of the underlying dynamics ${\mathit{d}}_{\mathit{L}}$. The embedding theorem only guarantees that the attractor of the system is fully unfolded using ${\mathit{d}}_{\mathit{E}}$ greater than 2${\mathit{d}}_{\mathit{A}}$, with ${\mathit{d}}_{\mathit{A}}$ the fractal attractor dimension. Using the idea of local false nearest neighbors, we discuss methods for determining the integer-valued ${\mathit{d}}_{\mathit{L}}$.

Local false nearest neighbors and dynamical dimensions from observed chaotic data.

We present a computational method to determine if an observed time series possesses structure statistically distinguishable from high-dimensional linearly correlated noise, possibly with a nonwhite spectrum. This method should be useful in identifying deterministic chaos in natural signals with broadband power spectra, and is capable of distinguishing between chaos and a random process that has the same power spectrum. The method compares nonlinear predictability of the given data to an ensemble of random control data sets

Matthew B. Kennel

Papers

Determining embedding dimension for phase-space reconstruction using a geometrical construction

Local false nearest neighbors and dynamical dimensions from observed chaotic data

Observing and modeling nonlinear dynamics in an internal combustion engine

Local false nearest neighbors and dynamical dimensions from observed chaotic data.

Method to distinguish possible chaos from colored noise and to determine embedding parameters