Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.

The Nature of Statistical Learning Theory

In this tutorial we give an overview of the basic ideas underlying Support Vector (SV) machines for function estimation. Furthermore, we include a summary of currently used algorithms for training SV machines, covering both the quadratic (or convex) programming part and advanced methods for dealing with large datasets. Finally, we mention some modifications and extensions that have been applied to the standard SV algorithm, and discuss the aspect of regularization from a SV perspective.

/pdf/a-tutorial-on-support-vector-regression-10mfvogjkf.pdf

A tutorial on support vector regression

The use of randomness is now an accepted tool in Theoretical Computer Science but not everyone is aware of the underpinnings of this methodology in Combinatorics - particularly, in what is now called the probabilistic Method as developed primarily by Paul Erdoős over the past half century. Here I will explore a particular set of problems - all dealing with “good” colorings of an underlying set of points relative to a given family of sets. A central point will be the evolution of these problems from the purely existential proofs of Erdős to the algorithmic aspects of much interest to this audience.

/pdf/the-probabilistic-method-5g63k0itke.pdf

The Probabilistic Method

Machine learning is one of the fastest growing areas of computer science, with far-reaching applications. The aim of this textbook is to introduce machine learning, and the algorithmic paradigms it offers, in a principled way. The book provides an extensive theoretical account of the fundamental ideas underlying machine learning and the mathematical derivations that transform these principles into practical algorithms. Following a presentation of the basics of the field, the book covers a wide array of central topics that have not been addressed by previous textbooks. These include a discussion of the computational complexity of learning and the concepts of convexity and stability; important algorithmic paradigms including stochastic gradient descent, neural networks, and structured output learning; and emerging theoretical concepts such as the PAC-Bayes approach and compression-based bounds. Designed for an advanced undergraduate or beginning graduate course, the text makes the fundamentals and algorithms of machine learning accessible to students and non-expert readers in statistics, computer science, mathematics, and engineering.

/pdf/understanding-machine-learning-from-theory-to-algorithms-5aeszqx33g.pdf

Understanding Machine Learning: From Theory To Algorithms

This important text and reference for researchers and students in machine learning, game theory, statistics and information theory offers a comprehensive treatment of the problem of predicting individual sequences. Unlike standard statistical approaches to forecasting, prediction of individual sequences does not impose any probabilistic assumption on the data-generating mechanism. Yet, prediction algorithms can be constructed that work well for all possible sequences, in the sense that their performance is always nearly as good as the best forecasting strategy in a given reference class. The central theme is the model of prediction using expert advice, a general framework within which many related problems can be cast and discussed. Repeated game playing, adaptive data compression, sequential investment in the stock market, sequential pattern analysis, and several other problems are viewed as instances of the experts' framework and analyzed from a common nonstochastic standpoint that often reveals new and intriguing connections.

/pdf/prediction-learning-and-games-31z1lqjpec.pdf

Prediction, learning, and games

Ramsey-type properties of relational structures

For a fixed integer n ϵ ω, a graph G of chromatic number greater than n is called persistent if for all n + 1-chromatic graphs H, the products G × H are n + 1-chromatic graphs. Wheter all graphs of chromatic number greater than n are persistent is a long-standing open problem due to Hedetniemi. We call a graph G strongly persistent if G is persistent and the Hajos sum of G with any other persistent graph H is still persistent. This paper extends the class of known persistent graphs by proving the following results: If G is constructed from copies of Kn+1 by Hajos sums, adding vertices and edges and at most one contraction of nonadjacent vertices, then G is strongly persistent. © 1929 John Wiley & Sons, Inc.

An approach to Hedetniemi's conjecture

Abstract A metric space $\text{M=}\left( M;\text{d} \right)$ is homogeneous if for every isometry $f$ of a finite subspace of $\text{M}$ to a subspace of $\text{M}$ there exists an isometry of $\text{M}$ onto $\text{M}$ extending $f$ . The space $\text{M}$ is universal if it isometrically embeds every finite metric space $\text{F}$ with $\text{dist}\left( \text{F} \right)\subseteq \text{dist}\left( \text{M} \right)$ (with $\text{dist}\left( \text{M} \right)$ being the set of distances between points in $\text{M}$ ). A metric space $U$ is a Urysohn metric space if it is homogeneous, universal, separable, and complete. (We deduce as a corollary that a Urysohn metric space $U$ isometrically embeds every separable metric space $\text{M}$ with $\text{dist}\left( \text{M} \right)\subseteq \text{dist}\left( U \right)$ .) The main results are: (1) A characterization of the sets $\text{dist}\left( U \right)$ for Urysohn metric spaces $U$ . (2) If $R$ is the distance set of a Urysohn metric space and $\text{M}$ and $\text{N}$ are two metric spaces, of any cardinality with distances in $R$ , then they amalgamate disjointly to a metric space with distances in $R$ . (3) The completion of every homogeneous, universal, separable metric space $\text{M}$ is homogeneous.

Distance Sets of Urysohn Metric Spaces

Set-Homogeneous Graphs

We will prove that for every colouring of the edges of the Rado graph,ℛ (that is the countable homogeneous graph), with finitely many coulours, it contains an isomorphic copy whose edges are coloured with at most two of the colours. It was known [4] that there need not be a copy whose edges are coloured with only one of the colours. For the proof we use the lexicographical order on the vertices of the Rado graph, defined by Erdos, Hajnal and Posa. Using the result we are able to describe a “Ramsey basis” for the class of Rado graphs whose edges are coloured with at most a finite number,r, of colours. This answers an old question of M. Pouzet.

Norbert Sauer

Papers

Ramsey-type properties of relational structures

An approach to Hedetniemi's conjecture

Distance Sets of Urysohn Metric Spaces

Set-Homogeneous Graphs

Edge partitions of the rado graph