Open AccessProceedings Article
Escaping the Convex Hull with Extrapolated Vector Machines
Patrick Haffner
- Vol. 14, pp 753-760
TLDR
Extrapolated Vector Machines (XVMs) are proposed which rely on extrapolations outside these convex hulls to improve SVM generalization very significantly on the MNIST [7] OCR data.Abstract:
Maximum margin classifiers such as Support Vector Machines (SVMs) critically depends upon the convex hulls of the training samples of each class, as they implicitly search for the minimum distance between the convex hulls. We propose Extrapolated Vector Machines (XVMs) which rely on extrapolations outside these convex hulls. XVMs improve SVM generalization very significantly on the MNIST [7] OCR data. They share similarities with the Fisher discriminant: maximize the inter-class margin while minimizing the intra-class disparity.read more
Citations
More filters
Scaling learning algorithms towards AI
TL;DR: It is argued that deep architectures have the potential to generalize in non-local ways, i.e., beyond immediate neighbors, and that this is crucial in order to make progress on the kind of complex tasks required for artificial intelligence.
Posted Content
Out-of-Distribution Generalization via Risk Extrapolation (REx)
David Krueger,Ethan Caballero,Joern-Henrik Jacobsen,Amy Zhang,Jonathan Binas,Dinghuai Zhang,Rémi Le Priol,Aaron Courville +7 more
TL;DR: This work introduces the principle of Risk Extrapolation (REx), and shows conceptually how this principle enables extrapolation, and demonstrates the effectiveness and scalability of instantiations of REx on various OoD generalization tasks.
Book ChapterDOI
The huller: a simple and efficient online SVM
Antoine Bordes,Léon Bottou +1 more
TL;DR: A novel online kernel classifier algorithm that converges to the Hard Margin SVM solution that is attractive when one seeks a competitive classifier with large datasets and limited computing resources is proposed.
Journal Article
Maximum Relative Margin and Data-Dependent Regularization
TL;DR: This article identifies its sensitivity to affine transformations of the data and to directions with large data spread and proposes improvements that only require simple extensions to existing maximum margin formulations and preserve the computational efficiency of SVMs.
Proceedings ArticleDOI
Interpolation and extrapolation: Comparison of definitions and survey of algorithms for convex and concave hulls
TL;DR: This publication categorizes existing literature and gives guidelines to choose an appropriate definition of extrapolation for a present problem, and presents hull algorithms, from classic approaches to recent advances, to help the reader to solve a problem, which is affected by extrapolation.
References
More filters
Journal ArticleDOI
Gradient-based learning applied to document recognition
Yann LeCun,Léon Bottou,Léon Bottou,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio,Patrick Haffner +6 more
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Journal ArticleDOI
Support-Vector Networks
Corinna Cortes,Vladimir Vapnik +1 more
TL;DR: High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
Statistical learning theory
TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Proceedings Article
Large Margin DAGs for Multiclass Classification
TL;DR: An algorithm, DAGSVM, is presented, which operates in a kernel-induced feature space and uses two-class maximal margin hyperplanes at each decision-node of the DDAG, which is substantially faster to train and evaluate than either the standard algorithm or Max Wins, while maintaining comparable accuracy to both of these algorithms.
Support Vector Machines for Large-Scale Regression Problems
Ronan Collobert,Samy Bengio +1 more
TL;DR: In this paper, learning reference EPFL-REPORT-82604 is used to learn Reference EPFL this paper. But learning reference is not considered in this paper. http://publications.idiap.ch/downloads/reports/2000/rr00-17.pdf Record created on 2006-03-10, modified on 2017-05-10