The support-vector network is a new learning machine for two-group classification problems. The machine conceptually implements the following idea: input vectors are non-linearly mapped to a very high-dimension feature space. In this feature space a linear decision surface is constructed. Special properties of the decision surface ensures high generalization ability of the learning machine. The idea behind the support-vector network was previously implemented for the restricted case where the training data can be separated without errors. We here extend this result to non-separable training data.

High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated. We also compare the performance of the support-vector network to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.

/pdf/support-vector-networks-2jd9a1kl0z.pdf

Support-Vector Networks

The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and non-separable data, working through a non-trivial example in detail. We describe a mechanical analogy, and discuss when SVM solutions are unique and when they are global. We describe how support vector training can be practically implemented, and discuss in detail the kernel mapping technique which is used to construct SVM solutions which are nonlinear in the data. We show how Support Vector machines can have very large (even infinite) VC dimension by computing the VC dimension for homogeneous polynomial and Gaussian radial basis function kernels. While very high VC dimension would normally bode ill for generalization performance, and while at present there exists no theory which shows that good generalization performance is guaranteed for SVMs, there are several arguments which support the observed high accuracy of SVMs, which we review. Results of some experiments which were inspired by these arguments are also presented. We give numerous examples and proofs of most of the key theorems. There is new material, and I hope that the reader will find that even old material is cast in a fresh light.

/pdf/a-tutorial-on-support-vector-machines-for-pattern-5cn6x20psd.pdf

A Tutorial on Support Vector Machines for Pattern Recognition

A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented. The technique is applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions. The effective number of parameters is adjusted automatically to match the complexity of the problem. The solution is expressed as a linear combination of supporting patterns. These are the subset of training patterns that are closest to the decision boundary. Bounds on the generalization performance based on the leave-one-out method and the VC-dimension are given. Experimental results on optical character recognition problems demonstrate the good generalization obtained when compared with other learning algorithms.

/pdf/a-training-algorithm-for-optimal-margin-classifiers-40vzb5mn76.pdf

A training algorithm for optimal margin classifiers

A new method for performing a nonlinear form of principal component analysis is proposed. By the use of integral operator kernel functions, one can efficiently compute principal components in high-dimensional feature spaces, related to input space by some nonlinear map—for instance, the space of all possible five-pixel products in 16 × 16 images. We give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition.

/pdf/nonlinear-component-analysis-as-a-kernel-eigenvalue-problem-3oz94lp1cq.pdf

Nonlinear component analysis as a kernel eigenvalue problem

Snakes, or active contours, are used extensively in computer vision and image processing applications, particularly to locate object boundaries. Problems associated with initialization and poor convergence to boundary concavities, however, have limited their utility. This paper presents a new external force for active contours, largely solving both problems. This external force, which we call gradient vector flow (GVF), is computed as a diffusion of the gradient vectors of a gray-level or binary edge map derived from the image. It differs fundamentally from traditional snake external forces in that it cannot be written as the negative gradient of a potential function, and the corresponding snake is formulated directly from a force balance condition rather than a variational formulation. Using several two-dimensional (2-D) examples and one three-dimensional (3-D) example, we show that GVF has a large capture range and is able to move snakes into boundary concavities.

/pdf/snakes-shapes-and-gradient-vector-flow-29x5p2ndjo.pdf

Snakes, shapes, and gradient vector flow

Partial table of contents: THE ALGEBRA OF LINEAR TRANSFORMATIONS AND QUADRATIC FORMS. Transformation to Principal Axes of Quadratic and Hermitian Forms. Minimum-Maximum Property of Eigenvalues. SERIES EXPANSION OF ARBITRARY FUNCTIONS. Orthogonal Systems of Functions. Measure of Independence and Dimension Number. Fourier Series. Legendre Polynomials. LINEAR INTEGRAL EQUATIONS. The Expansion Theorem and Its Applications. Neumann Series and the Reciprocal Kernel. The Fredholm Formulas. THE CALCULUS OF VARIATIONS. Direct Solutions. The Euler Equations. VIBRATION AND EIGENVALUE PROBLEMS. Systems of a Finite Number of Degrees of Freedom. The Vibrating String. The Vibrating Membrane. Green's Function (Influence Function) and Reduction of Differential Equations to Integral Equations. APPLICATION OF THE CALCULUS OF VARIATIONS TO EIGENVALUE PROBLEMS. Completeness and Expansion Theorems. Nodes of Eigenfunctions. SPECIAL FUNCTIONS DEFINED BY EIGENVALUE PROBLEMS. Bessel Functions. Asymptotic Expansions. Additional Bibliography. Index.

Methods of Mathematical Physics

VIII uber den Inhalt im einzelnen unterrichtet das ausfuhrliche Ver zeichnis. Zur Form ist etwas Grundsatzliches zu sagen: Das klassische Ideal einer gewissermassen atomistischen Auffassung der Mathematik ver langt, den Stoff in Form von Voraussetzungen, Satzen und Beweisen zu kondensieren. Dabei ist der innere Zusammenhang und die Motivierung der Theorie nicht unmittelbar Gegenstand der Darstellung. In kom plementarer Weise kann man ein mathematisches Gebiet als stetiges Gewebe von Zusammenhangen betrachten, bei dessen Beschreibung die Methode und die Motivierung in den Vordergrund treten und die Kri stallisierung der Einsichten in isolierte scharf umrissene Satze erst eine sekundare Rolle spielt. Wo eine Synthese beider Auffassungen untunlich schien, habe ich den zweiten Gesichtspunkt bevorzugt. New Rochelle, New York, 24. Oktober 1937. R. Courant. Inhaltsverzeichnis. Erstes Kapitel. Vorbereitung. - Grundbegriffe. I. Orientierung uber die Mannigfaltigkeit der Losungen 2 1. Beispiele S. 2. - 2. Differentialgleichungen zu gegebenen Funk tionenscharen und -familien S. 7. 2. Systeme von Differentialgleichungen 10 1. Problem der Aquivalenz von Systemen und einzelnen Differential 2. Bestimmte, uberbestimmte, unterbestimmte gleichungen S. 10. - Systeme S. 12. J. Integrationsmethoden bei speziellen Differentialgleichungen. . . . . . 14 1. Separation der Variablen S. 14. - 2. Erzeugung weiterer Losungen durch Superposition. Grundlosung der Warmeleitung. Poissons Integral S.16. 4. Geometrische Deutung einer partiellen Differentialgleichung erster Ord nung mit zwei unabhangigen Variablen. Das vollstandige Integral . . 18 1. Die geometrische Deutung einer partiellen Differentialgleichung erster Ordnung S. 18. - 2. Das vollstandige Integral S. 19. - 3. Singulare Integrale S. 20."

Methoden der mathematischen Physik

Methods of Mathematical Physics, Vol. I

The simplest curves and surfaces Regular systems of points Projective configurations Differential geometry Kinematics Topology Index.

David Hilbert

Papers

Methods of Mathematical Physics

Methods of Mathematical Physics

Methoden der mathematischen Physik

Methods of Mathematical Physics, Vol. I

Geometry and the imagination