A comprehensive and self-contained introduction to Gaussian processes, which provide a principled, practical, probabilistic approach to learning in kernel machines. Gaussian processes (GPs) provide a principled, practical, probabilistic approach to learning in kernel machines. GPs have received increased attention in the machine-learning community over the past decade, and this book provides a long-needed systematic and unified treatment of theoretical and practical aspects of GPs in machine learning. The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics. The book deals with the supervised-learning problem for both regression and classification, and includes detailed algorithms. A wide variety of covariance (kernel) functions are presented and their properties discussed. Model selection is discussed both from a Bayesian and a classical perspective. Many connections to other well-known techniques from machine learning and statistics are discussed, including support-vector machines, neural networks, splines, regularization networks, relevance vector machines and others. Theoretical issues including learning curves and the PAC-Bayesian framework are treated, and several approximation methods for learning with large datasets are discussed. The book contains illustrative examples and exercises, and code and datasets are available on the Web. Appendixes provide mathematical background and a discussion of Gaussian Markov processes.

/pdf/gaussian-processes-for-machine-learning-40fp9c190d.pdf

Gaussian Processes for Machine Learning

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

This survey provides an overview of higher-order tensor decompositions, their applications, and available software. A tensor is a multidimensional or $N$-way array. Decompositions of higher-order tensors (i.e., $N$-way arrays with $N \geq 3$) have applications in psycho-metrics, chemometrics, signal processing, numerical linear algebra, computer vision, numerical analysis, data mining, neuroscience, graph analysis, and elsewhere. Two particular tensor decompositions can be considered to be higher-order extensions of the matrix singular value decomposition: CANDECOMP/PARAFAC (CP) decomposes a tensor as a sum of rank-one tensors, and the Tucker decomposition is a higher-order form of principal component analysis. There are many other tensor decompositions, including INDSCAL, PARAFAC2, CANDELINC, DEDICOM, and PARATUCK2 as well as nonnegative variants of all of the above. The N-way Toolbox, Tensor Toolbox, and Multilinear Engine are examples of software packages for working with tensors.

/pdf/tensor-decompositions-and-applications-46vl2ih5gn.pdf

Tensor Decompositions and Applications

Data Mining Practical Machine Learning Tools and Techniques

Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

Machine Learning : A Probabilistic Perspective

We discuss a multilinear generalization of the singular value decomposition. There is a strong analogy between several properties of the matrix and the higher-order tensor decomposition; uniqueness, link with the matrix eigenvalue decomposition, first-order perturbation effects, etc., are analyzed. We investigate how tensor symmetries affect the decomposition and propose a multilinear generalization of the symmetric eigenvalue decomposition for pair-wise symmetric tensors.

/pdf/a-multilinear-singular-value-decomposition-5egsfnmazm.pdf

A Multilinear Singular Value Decomposition

Support Vector Machines Basic Methods of Least Squares Support Vector Machines Bayesian Inference for LS-SVM Models Robustness Large Scale Problems LS-SVM for Unsupervised Learning LS-SVM for Recurrent Networks and Control.

/pdf/least-squares-support-vector-machines-2mmqf1mu71.pdf

Least Squares Support Vector Machines

Subspace Identification for Linear Systems focuses on the theory, implementation and applications of subspace identification algorithms for linear time-invariant finitedimensional dynamical systems. These algorithms allow for a fast, straightforward
and accurate determination of linear multivariable models from measured inputoutput data. The theory of subspace identification algorithms is presented in detail. Several chapters are devoted to deterministic, stochastic and combined deterministicstochastic
subspace identification algorithms. For each case, the geometric properties are stated in a main 'subspace' Theorem. Relations to existing algorithms and literature are explored, as are the interconnections between different subspace algorithms. The subspace identification theory is linked to the theory of frequency weighted model reduction, which leads to new interpretations and insights. The implementation of subspace identification algorithms is discussed in terms of the robust and computationally efficient RQ and singular value decompositions, which are well-established algorithms from numerical linear algebra. The algorithms are implemented in combination with a whole set of classical identification algorithms,processing and validation tools in Xmath's ISID, a commercially available graphical user interface toolbox. The basic subspace algorithms in the book are also implemented in a set of MATLABÂ® files accompanying the book. An application of ISID to an industrial glass tube manufacturing process is presented in detail, illustrating the power and user-friendliness of the subspace identification
algorithms and of their implementation in ISID. The identified model allows for an optimal control of the process, leading to a significant enhancement of the production quality. The applicability of subspace identification algorithms in industry is further illustrated with the application of the MATLABÂ® files to ten practical problems. Since all necessary data and MATLABÂ® files are included, the reader can easily step through these applications, and thus get more insight in the algorithms.
Subspace Identification for Linear Systems is an important reference for all researchers in system theory, control theory, signal processing, automization,mechatronics, chemical, electrical, mechanical and aeronautical engineering.

Subspace Identification for Linear Systems: Theory - Implementation - Applications

N4SID: subspace algorithms for the identification of combined deterministic-stochastic systems

Summary:biomaRt is a new Bioconductor package that integrates BioMart data resources with data analysis software in Bioconductor. It can annotate a wide range of gene or gene product identifiers (e.g. Entrez-Gene and Affymetrix probe identifiers) with information such as gene symbol, chromosomal coordinates, Gene Ontology and OMIM annotation. Furthermore biomaRt enables retrieval of genomic sequences and single nucleotide polymorphism information, which can be used in data analysis. Fast and up-to-date data retrieval is possible as the package executes direct SQL queries to the BioMart databases (e.g. Ensembl). The biomaRt package provides a tight integration of large, public or locally installed BioMart databases with data analysis in Bioconductor creating a powerful environment for biological data mining.

Availability:http://www.bioconductor.org. LGPL

Contact: steffen.durinck@esat.kuleuven.ac.be

/pdf/biomart-and-bioconductor-a-powerful-link-between-biological-113p4fvnvy.pdf

Bart De Moor

Papers

A Multilinear Singular Value Decomposition

Least Squares Support Vector Machines

Subspace Identification for Linear Systems: Theory - Implementation - Applications

N4SID: subspace algorithms for the identification of combined deterministic-stochastic systems

BioMart and Bioconductor: a powerful link between biological databases and microarray data analysis