LIBSVM: A library for support vector machines

doi:10.1145/1961189.1961199

Home
/
Papers
/
LIBSVM: A library for support vector machines

Journal Article•DOI•

LIBSVM: A library for support vector machines

Chih-Chung Chang¹, Chih-Jen Lin¹•Institutions (1)

National Taiwan University¹

06 May 2011-ACM Transactions on Intelligent Systems and Technology (ACM)-Vol. 2, Iss: 3, pp 27

TL;DR: Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.

read less

Abstract: LIBSVM is a library for Support Vector Machines (SVMs). We have been actively developing this package since the year 2000. The goal is to help users to easily apply SVM to their applications. LIBSVM has gained wide popularity in machine learning and many other areas. In this article, we present all implementation details of LIBSVM. Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.

...read moreread less

Content maybe subject to copyright Report

Citations

PDF

Open Access

More filters

Posted Content•

How Powerful are Graph Neural Networks

[...]

Keyulu Xu¹, Weihua Hu², Jure Leskovec², Stefanie Jegelka¹•Institutions (2)

Massachusetts Institute of Technology¹, Stanford University²

01 Oct 2018-arXiv: Learning

TL;DR: This work characterize the discriminative power of popular GNN variants, such as Graph Convolutional Networks and GraphSAGE, and show that they cannot learn to distinguish certain simple graph structures, and develops a simple architecture that is provably the most expressive among the class of GNNs.

...read moreread less

Abstract: Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs. GNNs follow a neighborhood aggregation scheme, where the representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes. Many GNN variants have been proposed and have achieved state-of-the-art results on both node and graph classification tasks. However, despite GNNs revolutionizing graph representation learning, there is limited understanding of their representational properties and limitations. Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures. Our results characterize the discriminative power of popular GNN variants, such as Graph Convolutional Networks and GraphSAGE, and show that they cannot learn to distinguish certain simple graph structures. We then develop a simple architecture that is provably the most expressive among the class of GNNs and is as powerful as the Weisfeiler-Lehman graph isomorphism test. We empirically validate our theoretical findings on a number of graph classification benchmarks, and demonstrate that our model achieves state-of-the-art performance.

...read moreread less

281 citations

Cites methods from "LIBSVM: A library for support vecto..."

...…GNNs above with a number of state-of-the-art baselines for graph classification: (1) the WL subtree kernel (Shervashidze et al., 2011), where C-SVM (Chang & Lin, 2011) was used as a classifier; the hyper-parameters we tune are C of the SVM and the number of WL iterations ∈ {1, 2, . . . , 6}; (2)…...
[...]
...We compare the GNNs above with a number of state-of-the-art baselines for graph classification: (1) the WL subtree kernel (Shervashidze et al., 2011), where C-SVM (Chang & Lin, 2011) was used as a classifier; the hyper-parameters we tune are C of the SVM and the number of WL iterations ∈ {1, 2, . . . , 6}; (2) state-of-the-art deep learning architectures, i.e., Diffusionconvolutional neural networks (DCNN) (Atwood & Towsley, 2016), PATCHY-SAN (Niepert et al., 2016) and Deep Graph CNN (DGCNN) (Zhang et al., 2018); (3) Anonymous Walk Embeddings (AWL) (Ivanov & Burnaev, 2018)....
[...]
...Following (Yanardag & Vishwanathan, 2015; Niepert et al., 2016), we perform 10-fold crossvalidation with LIB-SVM (Chang & Lin, 2011)....
[...]

Journal Article•DOI•

Direct Remaining Useful Life Estimation Based on Support Vector Regression

[...]

Racha Khelif, Brigitte Chebel-Morello, Simon Malinowski¹, Emna Laajili, Farhat Fnaiech², Noureddine Zerhouni - Show less +2 more•Institutions (2)

University of Rennes¹, Tunis University²

01 Mar 2017-IEEE Transactions on Industrial Electronics

TL;DR: Experimental results show that the performance of the proposed method is competitive with other existing approaches and has a positive impact on the accuracy of the prediction while reducing the computational time compared to existing indirect RUL prediction methods.

...read moreread less

Abstract: Prognostics is a major activity in the field of prognostics and health management It aims at increasing the reliability and safety of systems while reducing the maintenance cost by providing an estimate of the current health status and remaining useful life (RUL) Classical RUL estimation techniques are usually composed of different steps: estimations of a health indicator, degradation states, a failure threshold, and finally the RUL In this work, a procedure that is able to estimate the RUL of equipment directly from sensor values without the need for estimating degradation states or a failure threshold is developed A direct relation between sensor values or health indicators is modeled using a support vector regression Using this procedure, the RUL can be estimated at any time instant of the degradation process In addition, an offline wrapper variable selection is applied before training the prediction model This step has a positive impact on the accuracy of the prediction while reducing the computational time compared to existing indirect RUL prediction methods To assess the performance of the proposed approach, the Turbofan dataset, widely considered in the literature, is used Experimental results show that the performance of the proposed method is competitive with other existing approaches

...read moreread less

281 citations

Cites methods from "LIBSVM: A library for support vecto..."

...The described approach was implemented using “LibSVM” [38] and applied on Turbofan engine degradation dataset available at the NASA prognostics data repository1 [39]....
[...]

Journal Article•DOI•

Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding

[...]

Lefei Zhang¹, Qian Zhang², Liangpei Zhang¹, Dacheng Tao³, Xin Huang¹, Bo Du¹ - Show less +2 more•Institutions (3)

Wuhan University¹, Samsung², University of Technology, Sydney³

01 Oct 2015-Pattern Recognition

TL;DR: This paper provides an efficient EMR-SLRA optimization procedure to obtain the output feature embedding and experiments on the pattern recognition applications confirm the effectiveness of the EMR -SLRA algorithm compare with some other multiview feature dimensionality reduction approaches.

...read moreread less

280 citations

Journal Article•DOI•

Reduced Support Vector Machines: A Statistical Theory

[...]

Yuh-Jye Lee¹, Su-Yun Huang²•Institutions (2)

National Taiwan University of Science and Technology¹, Academia Sinica²

01 Jan 2007-IEEE Transactions on Neural Networks

TL;DR: This paper study the RSVM from the viewpoint of sampling design, its robustness, and the spectral analysis of the reduced kernel, which indicates that the approximation kernels can retain most of the relevant information for learning tasks in the full kernel.

...read moreread less

Abstract: In dealing with large data sets, the reduced support vector machine (RSVM) was proposed for the practical objective to overcome some computational difficulties as well as to reduce the model complexity. In this paper, we study the RSVM from the viewpoint of sampling design, its robustness, and the spectral analysis of the reduced kernel. We consider the nonlinear separating surface as a mixture of kernels. Instead of a full model, the RSVM uses a reduced mixture with kernels sampled from certain candidate set. Our main results center on two major themes. One is the robustness of the random subset mixture model. The other is the spectral analysis of the reduced kernel. The robustness is judged by a few criteria as follows: 1) model variation measure; 2) model bias (deviation) between the reduced model and the full model; and 3) test power in distinguishing the reduced model from the full one. For the spectral analysis, we compare the eigenstructures of the full kernel matrix and the approximation kernel matrix. The approximation kernels are generated by uniform random subsets. The small discrepancies between them indicate that the approximation kernels can retain most of the relevant information for learning tasks in the full kernel. We focus on some statistical theory of the reduced set method mainly in the context of the RSVM. The use of a uniform random subset is not limited to the RSVM. This approach can act as a supplemental algorithm on top of a basic optimization algorithm, wherein the actual optimization takes place on the subset-approximated data. The statistical properties discussed in this paper are still valid

...read moreread less

280 citations

Journal Article•DOI•

Dissecting the Phenotypic Components of Crop Plant Growth and Drought Responses Based on High-Throughput Image Analysis

[...]

Dijun Chen¹, Dijun Chen², Kerstin Neumann¹, Swetlana Friedel¹, Benjamin Kilian¹, Ming Chen², Thomas Altmann¹, Christian Klukas¹ - Show less +4 more•Institutions (2)

Leibniz Association¹, Zhejiang University²

29 Jan 2015-The Plant Cell

TL;DR: A comprehensive framework for high-throughput phenotype data analysis in plants is developed, which enables the extraction of an extensive list of phenotypic traits from nondestructive plant imaging over time and is promising for subsequent genetic mapping to uncover the genetic basis of complex agronomic traits.

...read moreread less

Abstract: Significantly improved crop varieties are urgently needed to feed the rapidly growing human population under changing climates. While genome sequence information and excellent genomic tools are in place for major crop species, the systematic quantification of phenotypic traits or components thereof in a high-throughput fashion remains an enormous challenge. In order to help bridge the genotype to phenotype gap, we developed a comprehensive framework for high-throughput phenotype data analysis in plants, which enables the extraction of an extensive list of phenotypic traits from nondestructive plant imaging over time. As a proof of concept, we investigated the phenotypic components of the drought responses of 18 different barley (Hordeum vulgare) cultivars during vegetative growth. We analyzed dynamic properties of trait expression over growth time based on 54 representative phenotypic features. The data are highly valuable to understand plant development and to further quantify growth and crop performance features. We tested various growth models to predict plant biomass accumulation and identified several relevant parameters that support biological interpretation of plant growth and stress tolerance. These image-based traits and model-derived parameters are promising for subsequent genetic mapping to uncover the genetic basis of complex agronomic traits. Taken together, we anticipate that the analytical framework and analysis results presented here will be useful to advance our views of phenotypic trait components underlying plant development and their responses to environmental cues.

...read moreread less

280 citations

Cites methods from "LIBSVM: A library for support vecto..."

...Based on their phenotypic traits (features), plants from the same genotype were classified into control and stress groups (Supplemental Figure 5A), using the pairwise classification strategy of the SVM algorithm as provided by the libsvm library (Chang and Lin, 2011) via the R package e1071....
[...]
...…SVM Based on their phenotypic traits (features), plants from the same genotype were classified into control and stress groups (Supplemental Figure 5A), using the pairwise classification strategy of the SVM algorithm as provided by the libsvm library (Chang and Lin, 2011) via the R package e1071....
[...]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
…
112
113
114
115
116
117
118
…
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200

Collapse

References

PDF

Open Access

More filters

Journal Article•DOI•

Support-Vector Networks

[...]

Corinna Cortes¹, Vladimir Vapnik¹•Institutions (1)

Bell Labs¹

15 Sep 1995-Machine Learning

TL;DR: High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.

...read moreread less

Abstract: The support-vector network is a new learning machine for two-group classification problems. The machine conceptually implements the following idea: input vectors are non-linearly mapped to a very high-dimension feature space. In this feature space a linear decision surface is constructed. Special properties of the decision surface ensures high generalization ability of the learning machine. The idea behind the support-vector network was previously implemented for the restricted case where the training data can be separated without errors. We here extend this result to non-separable training data. High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated. We also compare the performance of the support-vector network to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.

...read moreread less

37,861 citations

"LIBSVM: A library for support vecto..." refers background in this paper

...{1,-1}, C-SVC [Boser et al. 1992; Cortes and Vapnik 1995] solves 4LIBSVM Tools: http://www.csie.ntu.edu.tw/~cjlin/libsvmtools. the following primal optimization problem: l t min 1 w T w +C .i (1) w,b,. 2 i=1 subject to yi(w T f(xi) +b) =1 -.i, .i =0,i =1,...,l, where f(xi)maps xi into a…...
[...]

Statistical learning theory

[...]

Vladimir Vapnik

01 Jan 1998

TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.

...read moreread less

Abstract: A comprehensive look at learning and generalization theory. The statistical theory of learning and generalization concerns the problem of choosing desired functions on the basis of empirical data. Highly applicable to a variety of computer science and robotics fields, this book offers lucid coverage of the theory as a whole. Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.

...read moreread less

26,531 citations

"LIBSVM: A library for support vecto..." refers background in this paper

...Under given parameters C > 0and E> 0, the standard form of support vector regression [Vapnik 1998] is ll tt 1 T min w w + C .i + C .i * w,b,.,. * 2 i=1 i=1 subject to w T f(xi) + b- zi = E + .i, zi - w T f(xi) - b = E + .i * , * .i,.i = 0,i = 1,...,l....
[...]
...It can be clearly seen that C-SVC and one-class SVM are already in the form of problem (11)....
[...]
..., l, in two classes, and a vector y ∈ Rl such that yi ∈ {1,−1}, C-SVC (Cortes and Vapnik, 1995; Vapnik, 1998) solves the following primal problem:...
[...]
...Then, according to the SVM formulation, svm train one calls a corresponding subroutine such as solve c svc for C-SVC and solve nu svc for ....
[...]
...Note that b of C-SVC and E-SVR plays the same role as -. in one-class SVM, so we de.ne ....
[...]

Proceedings Article•DOI•

A training algorithm for optimal margin classifiers

[...]

Bernhard E. Boser¹, Isabelle Guyon², Vladimir Vapnik²•Institutions (2)

University of California, Berkeley¹, Bell Labs²

01 Jul 1992

TL;DR: A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented, applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions.

...read moreread less

Abstract: A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented. The technique is applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions. The effective number of parameters is adjusted automatically to match the complexity of the problem. The solution is expressed as a linear combination of supporting patterns. These are the subset of training patterns that are closest to the decision boundary. Bounds on the generalization performance based on the leave-one-out method and the VC-dimension are given. Experimental results on optical character recognition problems demonstrate the good generalization obtained when compared with other learning algorithms.

...read moreread less

11,211 citations

"LIBSVM: A library for support vecto..." refers background in this paper

...It can be clearly seen that C-SVC and one-class SVM are already in the form of problem (11)....
[...]
...Then, according to the SVM formulation, svm train one calls a corresponding subroutine such as solve c svc for C-SVC and solve nu svc for ....
[...]
...Note that b of C-SVC and E-SVR plays the same role as -. in one-class SVM, so we de.ne ....
[...]
...In Section 2, we describe SVM formulations supported in LIBSVM: C-Support Vector Classi.cation (C-SVC), ....
[...]
...{1,-1}, C-SVC [Boser et al. 1992; Cortes and Vapnik 1995] solves 4LIBSVM Tools: http://www.csie.ntu.edu.tw/~cjlin/libsvmtools. the following primal optimization problem: l t min 1 w T w +C .i (1) w,b,. 2 i=1 subject to yi(w T f(xi) +b) =1 -.i, .i =0,i =1,...,l, where f(xi)maps xi into a higher-dimensional space and C > 0 is the regularization parameter....
[...]

A Practical Guide to Support Vector Classication

[...]

Hsu Chih-Wei, Chih-Chung Chang¹, Chih-Jen Lin•Institutions (1)

National Taiwan University¹

01 Jan 2008

TL;DR: A simple procedure is proposed, which usually gives reasonable results and is suitable for beginners who are not familiar with SVM.

...read moreread less

Abstract: Support vector machine (SVM) is a popular technique for classication. However, beginners who are not familiar with SVM often get unsatisfactory results since they miss some easy but signicant steps. In this guide, we propose a simple procedure, which usually gives reasonable results.

...read moreread less

7,069 citations

"LIBSVM: A library for support vecto..." refers methods in this paper

...A Simple Example of Running LIBSVM While detailed instructions of using LIBSVM are available in the README file of the package and the practical guide by Hsu et al. [2003], here we give a simple example....
[...]
...For instructions of using LIBSVM, see the README file included in the package, the LIBSVM FAQ,3 and the practical guide by Hsu et al. [2003]. LIBSVM supports the following learning tasks....
[...]

Journal Article•DOI•

A comparison of methods for multiclass support vector machines

[...]

Hsu Chih-Wei¹, Chih-Jen Lin¹•Institutions (1)

National Taiwan University¹

01 Mar 2002-IEEE Transactions on Neural Networks

TL;DR: Decomposition implementations for two "all-together" multiclass SVM methods are given and it is shown that for large problems methods by considering all data at once in general need fewer support vectors.

...read moreread less

Abstract: Support vector machines (SVMs) were originally designed for binary classification. How to effectively extend it for multiclass classification is still an ongoing research issue. Several methods have been proposed where typically we construct a multiclass classifier by combining several binary classifiers. Some authors also proposed methods that consider all classes at once. As it is computationally more expensive to solve multiclass problems, comparisons of these methods using large-scale problems have not been seriously conducted. Especially for methods solving multiclass SVM in one step, a much larger optimization problem is required so up to now experiments are limited to small data sets. In this paper we give decomposition implementations for two such "all-together" methods. We then compare their performance with three methods based on binary classifications: "one-against-all," "one-against-one," and directed acyclic graph SVM (DAGSVM). Our experiments indicate that the "one-against-one" and DAG methods are more suitable for practical use than the other methods. Results also show that for large problems methods by considering all data at once in general need fewer support vectors.

...read moreread less

6,562 citations