From neural networks to support vector machines(A)

TLDR: This paper is a tutorial in which the basic concepts of VC theory and the methodology of SVMs as applied to pattern recognition problems are reviewed.

Abstract: In the field of statistical pattern recognition, optimal classifiers may be designed theoretically based on the Bayesian decision rule, however, it is necessary for the implementation of the design to so1ve a more difficu1t problem of density estimation first The strategy adopted in HP neural networks is learning di-rectly front the measurement data( training samples), which is more efficient and effective Therefore the methodology of neural networks has been widely used in real life applications, but like other heuristic meth-ods, it lacks a solid theoretical foundation to direct engineering practice As the result of the breakthrough in the research of statistical inference, VC theory has been established and accepted as the modern statistical learning theory The behavior of neural networks may he explained by VC theory with mathematical rigor in addition, a more powerful learning method-the support vector machine has been constructed based on the theory and gained real life applications This paper is a tutorial in which the basic concepts of VC theory and the methodology of SVMs as applied to pattern recognition problems are reviewed