LIBSVM: A library for support vector machines

doi:10.1145/1961189.1961199

Home
/
Papers
/
LIBSVM: A library for support vector machines

Journal Article•DOI•

LIBSVM: A library for support vector machines

Chih-Chung Chang¹, Chih-Jen Lin¹•Institutions (1)

National Taiwan University¹

06 May 2011-ACM Transactions on Intelligent Systems and Technology (ACM)-Vol. 2, Iss: 3, pp 27

TL;DR: Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.

read less

Abstract: LIBSVM is a library for Support Vector Machines (SVMs). We have been actively developing this package since the year 2000. The goal is to help users to easily apply SVM to their applications. LIBSVM has gained wide popularity in machine learning and many other areas. In this article, we present all implementation details of LIBSVM. Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.

...read moreread less

Content maybe subject to copyright Report

Citations

PDF

Open Access

More filters

Journal Article•DOI•

Learning and Transferring Deep Joint Spectral–Spatial Features for Hyperspectral Classification

[...]

Jingxiang Yang¹, Yongqiang Zhao¹, Jonathan Cheung-Wai Chan²•Institutions (2)

Northwestern Polytechnical University¹, Vrije Universiteit Brussel²

16 May 2017-IEEE Transactions on Geoscience and Remote Sensing

TL;DR: Experiments demonstrate that the learned deep joint spectral–spatial features are discriminative, and competitive classification results can be achieved when compared with state-of-the-art methods.

...read moreread less

Abstract: Feature extraction is of significance for hyperspectral image (HSI) classification. Compared with conventional hand-crafted feature extraction, deep learning can automatically learn features with discriminative information. However, two issues exist in applying deep learning to HSIs. One issue is how to jointly extract spectral features and spatial features, and the other one is how to train the deep model when training samples are scarce. In this paper, a deep convolutional neural network with two-branch architecture is proposed to extract the joint spectral–spatial features from HSIs. The two branches of the proposed network are devoted to features from the spectral domain as well as the spatial domain. The learned spectral features and spatial features are then concatenated and fed to fully connected layers to extract the joint spectral–spatial features for classification. When the training samples are limited, we investigate the transfer learning to improve the performance. Low and mid-layers of the network are pretrained and transferred from other data sources; only top layers are trained with limited training samples extracted from the target scene. Experiments on Airborne Visible/Infrared Imaging Spectrometer and Reflective Optics System Imaging Spectrometer data demonstrate that the learned deep joint spectral–spatial features are discriminative, and competitive classification results can be achieved when compared with state-of-the-art methods. The experiments also reveal that the transferred features boost the classification performance.

...read moreread less

345 citations

Cites methods from "LIBSVM: A library for support vecto..."

...The SVM classification method is implemented with LIBSVM toolbox [50], the type of kernel function of SVM is radial basis function, and the regularization parameters are determined by fivefold crossvalidation in the range of [2−10, 2−9, ....
[...]

Proceedings Article•DOI•

Learning Community Embedding with Community Detection and Node Embedding on Graphs

[...]

Sandro Cavallari¹, Vincent W. Zheng², Hongyun Cai², Kevin Chen-Chuan Chang³, Erik Cambria¹ - Show less +1 more•Institutions (3)

Nanyang Technological University¹, Agency for Science, Technology and Research², University of Illinois at Urbana–Champaign³

06 Nov 2017

TL;DR: This paper studies an important yet largely under-explored setting of graph embedding, i.e., embedding communities instead of each individual nodes, and proposes a novel community embedding framework that jointly solves the three tasks together.

...read moreread less

Abstract: In this paper, we study an important yet largely under-explored setting of graph embedding, i.e., embedding communities instead of each individual nodes. We find that community embedding is not only useful for community-level applications such as graph visualization, but also beneficial to both community detection and node classification. To learn such embedding, our insight hinges upon a closed loop among community embedding, community detection and node embedding. On the one hand, node embedding can help improve community detection, which outputs good communities for fitting better community embedding. On the other hand, community embedding can be used to optimize the node embedding by introducing a community-aware high-order proximity. Guided by this insight, we propose a novel community embedding framework that jointly solves the three tasks together. We evaluate such a framework on multiple real-world datasets, and show that it improves graph visualization and outperforms state-of-the-art baselines in various application tasks, e.g., community detection and node classification.

...read moreread less

345 citations

Cites methods from "LIBSVM: A library for support vecto..."

...We use the remaining nodes, together with their labels, to train a classifier by LibSVM (c = 1 for all the methods) [6]....
[...]

Journal Article•DOI•

[...]

Hao Ding, Ichigaku Takigawa, Hiroshi Mamitsuka, Shanfeng Zhu

01 Sep 2014-Briefings in Bioinformatics

TL;DR: This article reviews the similarity-based machine learning methods for predicting drug-target interactions, which are state-of-the-art and have aroused great interest in bioinformatics.

...read moreread less

Abstract: Computationally predicting drug-target interactions is useful to select possible drug (or target) candidates for further biochemical verification. We focus on machine learning-based approaches, particularly similarity-based methods that use drug and target similarities, which show relationships among drugs and those among targets, respectively. These two similarities represent two emerging concepts, the chemical space and the genomic space. Typically, the methods combine these two types of similarities to generate models for predicting new drug-target interactions. This process is also closely related to a lot of work in pharmacogenomics or chemical biology that attempt to understand the relationships between the chemical and genomic spaces. This background makes the similarity-based approaches attractive and promising. This article reviews the similarity-based machine learning methods for predicting drug-target interactions, which are state-of-the-art and have aroused great interest in bioinformatics. We describe each of these methods briefly, and empirically compare these methods under a uniform experimental setting to explore their advantages and limitations.

...read moreread less

344 citations

Cites methods from "LIBSVM: A library for support vecto..."

...We used LIBSVM [64] for implementing BLM and PKM....
[...]

Journal Article•DOI•

Multi-Directional Multi-Level Dual-Cross Patterns for Robust Face Recognition

[...]

Changxing Ding¹, Jonghyun Choi², Dacheng Tao¹, Larry S. Davis²•Institutions (2)

University of Technology, Sydney¹, University of Maryland, College Park²

01 Mar 2016-IEEE Transactions on Pattern Analysis and Machine Intelligence

TL;DR: Experimental results indicate that DCP outperforms the state-of-the-art local descriptors for both face identification and face verification tasks and the best performance is achieved on the challenging LFW and FRGC 2.0 databases by deploying MDML-DCPs in a simple recognition scheme.

...read moreread less

Abstract: To perform unconstrained face recognition robust to variations in illumination, pose and expression, this paper presents a new scheme to extract “Multi-Directional Multi-Level Dual-Cross Patterns” (MDML-DCPs) from face images. Specifically, the MDML-DCPs scheme exploits the first derivative of Gaussian operator to reduce the impact of differences in illumination and then computes the DCP feature at both the holistic and component levels. DCP is a novel face image descriptor inspired by the unique textural structure of human faces. It is computationally efficient and only doubles the cost of computing local binary patterns, yet is extremely robust to pose and expression variations. MDML-DCPs comprehensively yet efficiently encodes the invariant characteristics of a face image from multiple levels into patterns that are highly discriminative of inter-personal differences but robust to intra-personal variations. Experimental results on the FERET, CAS-PERL-R1, FRGC 2.0, and LFW databases indicate that DCP outperforms the state-of-the-art local descriptors (e.g., LBP, LTP, LPQ, POEM, tLBP, and LGXP) for both face identification and face verification tasks. More impressively, the best performance is achieved on the challenging LFW and FRGC 2.0 databases by deploying MDML-DCPs in a simple recognition scheme.

...read moreread less

344 citations

Cites background from "LIBSVM: A library for support vecto..."

...There are only two remaining sets of parameters to tune over the four databases: the first is the sampling radii Rin and Rex of DCP, and the other is the cost parameter c of the linear SVM [39]....
[...]
...Moreover, with either linear SVM or score averaging, MDMLDCPs is able to achieve better performance than existing methods, and using both fusion methods the standard error of classification accuracy is consistently smaller than the other approaches....
[...]
...For face matching, multi-class classifiers have been used for face identification, such as the nearest neighbor (NN) classifier and the sparse representation classifier (SRC) [4], with two-class classifiers, such as support vector machine (SVM) and Bayesian analysis, being used for verification....
[...]
...We observe that: 1) The MDML-DCPs approach achieves superior performance on both Experiment 1 and 4 with the single descriptor DCP. Score fusion by linear SVM also works slightly better than score averaging....
[...]
...Both the PLDA classifier and the more recently proposed Joint Bayesian (JB) classifier [35] are tested for face matching, and both linear SVM and score averaging are tested for the score fusion....
[...]

Journal Article•DOI•

Classification of Hyperspectral Images by Exploiting Spectral–Spatial Information of Superpixel via Multiple Kernels

[...]

Leyuan Fang¹, Shutao Li¹, Wuhui Duan¹, Jinchang Ren, Jon Atli Benediktsson² - Show less +1 more•Institutions (2)

Hunan University¹, University of Iceland²

02 Jul 2015-IEEE Transactions on Geoscience and Remote Sensing

TL;DR: Experimental results on three widely used real HSIs indicate that the proposed SC-MK approach outperforms several well-known classification methods.

...read moreread less

Abstract: For the classification of hyperspectral images (HSIs), this paper presents a novel framework to effectively utilize the spectral–spatial information of superpixels via multiple kernels, which is termed as superpixel-based classification via multiple kernels (SC-MK). In the HSI, each superpixel can be regarded as a shape-adaptive region, which consists of a number of spatial neighboring pixels with very similar spectral characteristics. First, the proposed SC-MK method adopts an oversegmentation algorithm to cluster the HSI into many superpixels. Then, three kernels are separately employed for the utilization of the spectral information, as well as spatial information, within and among superpixels. Finally, the three kernels are combined together and incorporated into a support vector machine classifier. Experimental results on three widely used real HSIs indicate that the proposed SC-MK approach outperforms several well-known classification methods.

...read moreread less

344 citations

Cites methods from "LIBSVM: A library for support vecto..."

...The SVM classifier does not consider the spatial information, which was implemented with the spectral-only Gaussian kernel and functions in the library for support vector machines (LIBSVM) library [45]....
[...]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
…
83
84
85
86
87
88
89
…
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200

Collapse

References

PDF

Open Access

More filters

Journal Article•DOI•

Support-Vector Networks

[...]

Corinna Cortes¹, Vladimir Vapnik¹•Institutions (1)

Bell Labs¹

15 Sep 1995-Machine Learning

TL;DR: High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.

...read moreread less

Abstract: The support-vector network is a new learning machine for two-group classification problems. The machine conceptually implements the following idea: input vectors are non-linearly mapped to a very high-dimension feature space. In this feature space a linear decision surface is constructed. Special properties of the decision surface ensures high generalization ability of the learning machine. The idea behind the support-vector network was previously implemented for the restricted case where the training data can be separated without errors. We here extend this result to non-separable training data. High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated. We also compare the performance of the support-vector network to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.

...read moreread less

37,861 citations

"LIBSVM: A library for support vecto..." refers background in this paper

...{1,-1}, C-SVC [Boser et al. 1992; Cortes and Vapnik 1995] solves 4LIBSVM Tools: http://www.csie.ntu.edu.tw/~cjlin/libsvmtools. the following primal optimization problem: l t min 1 w T w +C .i (1) w,b,. 2 i=1 subject to yi(w T f(xi) +b) =1 -.i, .i =0,i =1,...,l, where f(xi)maps xi into a…...
[...]

Statistical learning theory

[...]

Vladimir Vapnik

01 Jan 1998

TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.

...read moreread less

Abstract: A comprehensive look at learning and generalization theory. The statistical theory of learning and generalization concerns the problem of choosing desired functions on the basis of empirical data. Highly applicable to a variety of computer science and robotics fields, this book offers lucid coverage of the theory as a whole. Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.

...read moreread less

26,531 citations

"LIBSVM: A library for support vecto..." refers background in this paper

...Under given parameters C > 0and E> 0, the standard form of support vector regression [Vapnik 1998] is ll tt 1 T min w w + C .i + C .i * w,b,.,. * 2 i=1 i=1 subject to w T f(xi) + b- zi = E + .i, zi - w T f(xi) - b = E + .i * , * .i,.i = 0,i = 1,...,l....
[...]
...It can be clearly seen that C-SVC and one-class SVM are already in the form of problem (11)....
[...]
..., l, in two classes, and a vector y ∈ Rl such that yi ∈ {1,−1}, C-SVC (Cortes and Vapnik, 1995; Vapnik, 1998) solves the following primal problem:...
[...]
...Then, according to the SVM formulation, svm train one calls a corresponding subroutine such as solve c svc for C-SVC and solve nu svc for ....
[...]
...Note that b of C-SVC and E-SVR plays the same role as -. in one-class SVM, so we de.ne ....
[...]

Proceedings Article•DOI•

A training algorithm for optimal margin classifiers

[...]

Bernhard E. Boser¹, Isabelle Guyon², Vladimir Vapnik²•Institutions (2)

University of California, Berkeley¹, Bell Labs²

01 Jul 1992

TL;DR: A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented, applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions.

...read moreread less

Abstract: A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented. The technique is applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions. The effective number of parameters is adjusted automatically to match the complexity of the problem. The solution is expressed as a linear combination of supporting patterns. These are the subset of training patterns that are closest to the decision boundary. Bounds on the generalization performance based on the leave-one-out method and the VC-dimension are given. Experimental results on optical character recognition problems demonstrate the good generalization obtained when compared with other learning algorithms.

...read moreread less

11,211 citations

"LIBSVM: A library for support vecto..." refers background in this paper

...It can be clearly seen that C-SVC and one-class SVM are already in the form of problem (11)....
[...]
...Then, according to the SVM formulation, svm train one calls a corresponding subroutine such as solve c svc for C-SVC and solve nu svc for ....
[...]
...Note that b of C-SVC and E-SVR plays the same role as -. in one-class SVM, so we de.ne ....
[...]
...In Section 2, we describe SVM formulations supported in LIBSVM: C-Support Vector Classi.cation (C-SVC), ....
[...]
...{1,-1}, C-SVC [Boser et al. 1992; Cortes and Vapnik 1995] solves 4LIBSVM Tools: http://www.csie.ntu.edu.tw/~cjlin/libsvmtools. the following primal optimization problem: l t min 1 w T w +C .i (1) w,b,. 2 i=1 subject to yi(w T f(xi) +b) =1 -.i, .i =0,i =1,...,l, where f(xi)maps xi into a higher-dimensional space and C > 0 is the regularization parameter....
[...]

A Practical Guide to Support Vector Classication

[...]

Hsu Chih-Wei, Chih-Chung Chang¹, Chih-Jen Lin•Institutions (1)

National Taiwan University¹

01 Jan 2008

TL;DR: A simple procedure is proposed, which usually gives reasonable results and is suitable for beginners who are not familiar with SVM.

...read moreread less

Abstract: Support vector machine (SVM) is a popular technique for classication. However, beginners who are not familiar with SVM often get unsatisfactory results since they miss some easy but signicant steps. In this guide, we propose a simple procedure, which usually gives reasonable results.

...read moreread less

7,069 citations

"LIBSVM: A library for support vecto..." refers methods in this paper

...A Simple Example of Running LIBSVM While detailed instructions of using LIBSVM are available in the README file of the package and the practical guide by Hsu et al. [2003], here we give a simple example....
[...]
...For instructions of using LIBSVM, see the README file included in the package, the LIBSVM FAQ,3 and the practical guide by Hsu et al. [2003]. LIBSVM supports the following learning tasks....
[...]

Journal Article•DOI•

A comparison of methods for multiclass support vector machines

[...]

Hsu Chih-Wei¹, Chih-Jen Lin¹•Institutions (1)

National Taiwan University¹

01 Mar 2002-IEEE Transactions on Neural Networks

TL;DR: Decomposition implementations for two "all-together" multiclass SVM methods are given and it is shown that for large problems methods by considering all data at once in general need fewer support vectors.

...read moreread less

Abstract: Support vector machines (SVMs) were originally designed for binary classification. How to effectively extend it for multiclass classification is still an ongoing research issue. Several methods have been proposed where typically we construct a multiclass classifier by combining several binary classifiers. Some authors also proposed methods that consider all classes at once. As it is computationally more expensive to solve multiclass problems, comparisons of these methods using large-scale problems have not been seriously conducted. Especially for methods solving multiclass SVM in one step, a much larger optimization problem is required so up to now experiments are limited to small data sets. In this paper we give decomposition implementations for two such "all-together" methods. We then compare their performance with three methods based on binary classifications: "one-against-all," "one-against-one," and directed acyclic graph SVM (DAGSVM). Our experiments indicate that the "one-against-one" and DAG methods are more suitable for practical use than the other methods. Results also show that for large problems methods by considering all data at once in general need fewer support vectors.

...read moreread less

6,562 citations