scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

An Investigation on Linear SVM and its Variants for Text Categorization

09 Feb 2010-pp 27-31
TL;DR: The results show that out of the eight implementations, SVMlin and Proximal SVM perform better in terms of consistent performance and reduced training time, however being an extremely simple algorithm with training time independent of the penalty parameter and the category for which training is being done, Proxinal SVM is appealing.
Abstract: Linear Support Vector Machines (SVMs) have been used successfully to classify text documents into set of concepts. With the increasing number of linear SVM formulations and decomposition algorithms publicly available, this paper performs a study on their efficiency and efficacy for text categorization tasks. Eight publicly available implementations are investigated in terms of Break Even Point (BEP), F1 measure, ROC plots, learning speed and sensitivity to penalty parameter, based on the experimental results on two benchmark text corpuses. The results show that out of the eight implementations, SVMlin and Proximal SVM perform better in terms of consistent performance and reduced training time. However being an extremely simple algorithm with training time independent of the penalty parameter and the category for which training is being done, Proximal SVM is appealing. We further investigated fuzzy proximal SVM on both the text corpuses; it showed improved generalization over proximal SVM.
Citations
More filters
Journal Article
Shi Bing1
TL;DR: Text categorization-assignment of natural language texts to one or more predefined categories based on their content-is an important component in many information organization and management tasks.
Abstract: Text categorization-assignment of natural language texts to one or more predefined categories based on their content-is an important component in many information organization and management tasks.Different automatic learning algorithms for text categori-zation have different classification accuracy.Very accurate text classifiers can be learned automatically from training examples.

384 citations

Proceedings ArticleDOI
01 Aug 2017
TL;DR: This paper provides a brief survey of the basic concepts and algorithms used for Machine Learning and its applications and introduces various learning modalities including supervised and unsupervised methods and deep learning paradigms.
Abstract: This paper provides a brief survey of the basic concepts and algorithms used for Machine Learning and its applications. We begin with a broader definition of machine learning and then introduce various learning modalities including supervised and unsupervised methods and deep learning paradigms. In the rest of the paper, we discuss applications of machine learning algorithms in various fields including pattern recognition, sensor networks, anomaly detection, Internet of Things (IoT) and health monitoring. In the final sections, we present some of the software tools and an extensive bibliography.

154 citations


Cites methods from "An Investigation on Linear SVM and ..."

  • ...The SVM algorithm is used in several applications including simple binary classification [135] text categorization [136-138], hand written digit recognition...

    [...]

01 Dec 2015
TL;DR: This paper has studied the existing work in the area of text classification and tried to summarize all existing information in a comprehensive and succinct manner to have a fair evaluation of the progress made in this field till date.
Abstract: Automated classification of text into predefined categories has always been considered as a vital method to manage and process a vast amount of documents in digital forms that are widespread and continuously increasing. This kind of web information, popularly known as the digital/electronic information is in the form of documents, conference material, publications, journals, editorials, web pages, e-mail etc. People largely access information from these online sources rather than being limited to archaic paper sources like books, magazines, newspapers etc. But the main problem is that this enormous information lacks organization which makes it difficult to manage. Text classification is recognized as one of the key techniques used for organizing such kind of digital data. In this paper we have studied the existing work in the area of text classification which will allow us to have a fair evaluation of the progress made in this field till date. We have investigated the papers to the best of our knowledge and have tried to summarize all existing information in a comprehensive and succinct manner. The studies have been summarized in a tabular form according to the publication year considering numerous key

51 citations

Journal ArticleDOI
TL;DR: A manifold regularized extreme learning machine is derived from the proposed framework, which maintains the properties of ELM and can be applicable to large-scale learning problems.
Abstract: Extreme learning machine (ELM) works for generalized single-hidden-layer feedforward networks (SLFNs), and its essence is that the hidden layer of SLFNs need not be tuned. But ELM only utilizes labeled data to carry out the supervised learning task. In order to exploit unlabeled data in the ELM model, we first extend the manifold regularization (MR) framework and then demonstrate the relation between the extended MR framework and ELM. Finally, a manifold regularized extreme learning machine is derived from the proposed framework, which maintains the properties of ELM and can be applicable to large-scale learning problems. Experimental results show that the proposed semi-supervised extreme learning machine is the most cost-efficient method. It tends to have better scalability and achieve satisfactory generalization performance at a relatively faster learning speed than traditional semi-supervised learning algorithms.

24 citations

Journal ArticleDOI
TL;DR: This study focuses live streaming of games and uses viewers’ text comments for experimental analysis, and proposes a text mining-based framework which includes Least Absolute Shrinkage and Selection Operator, Support Vector Machine-Recursive Feature Elimination, and Chi-square test to determine the important keywords of predicting the number of views in live streaming.

21 citations

References
More filters
Book
25 Oct 1999
TL;DR: This highly anticipated third edition of the most acclaimed work on data mining and machine learning will teach you everything you need to know about preparing inputs, interpreting outputs, evaluating results, and the algorithmic methods at the heart of successful data mining.
Abstract: Data Mining: Practical Machine Learning Tools and Techniques offers a thorough grounding in machine learning concepts as well as practical advice on applying machine learning tools and techniques in real-world data mining situations. This highly anticipated third edition of the most acclaimed work on data mining and machine learning will teach you everything you need to know about preparing inputs, interpreting outputs, evaluating results, and the algorithmic methods at the heart of successful data mining. Thorough updates reflect the technical changes and modernizations that have taken place in the field since the last edition, including new material on Data Transformations, Ensemble Learning, Massive Data Sets, Multi-instance Learning, plus a new version of the popular Weka machine learning software developed by the authors. Witten, Frank, and Hall include both tried-and-true techniques of today as well as methods at the leading edge of contemporary research. *Provides a thorough grounding in machine learning concepts as well as practical advice on applying the tools and techniques to your data mining projects *Offers concrete tips and techniques for performance improvement that work by transforming the input or output in machine learning methods *Includes downloadable Weka software toolkit, a collection of machine learning algorithms for data mining tasks-in an updated, interactive interface. Algorithms in toolkit cover: data pre-processing, classification, regression, clustering, association rules, visualization

20,196 citations

Book ChapterDOI
21 Apr 1998
TL;DR: This paper explores the use of Support Vector Machines for learning text classifiers from examples and analyzes the particular properties of learning with text data and identifies why SVMs are appropriate for this task.
Abstract: This paper explores the use of Support Vector Machines (SVMs) for learning text classifiers from examples. It analyzes the particular properties of learning with text data and identifies why SVMs are appropriate for this task. Empirical results support the theoretical findings. SVMs achieve substantial improvements over the currently best performing methods and behave robustly over a variety of different learning tasks. Furthermore they are fully automatic, eliminating the need for manual parameter tuning.

8,658 citations


"An Investigation on Linear SVM and ..." refers methods or result in this paper

  • ...Following [11], from the 50,216 documents in 1991, we used first 10,000 for training and second 10,000 for testing....

    [...]

  • ...These results go in line with BEP values presented on the same datasets by Joachims [11] and Dumais et al. [14]....

    [...]

  • ...These results go in line with BEP values presented on the same datasets by Joachims [11] and Dumais et al....

    [...]

Journal ArticleDOI
TL;DR: An algorithm for suffix stripping is described, which has been implemented as a short, fast program in BCPL, and performs slightly better than a much more elaborate system with which it has been compared.
Abstract: The automatic removal of suffixes from words in English is of particular interest in the field of information retrieval. An algorithm for suffix stripping is described, which has been implemented as a short, fast program in BCPL. Although simple, it performs slightly better than a much more elaborate system with which it has been compared. It effectively works by treating complex suffixes as compounds made up of simple suffixes, and removing the simple suffixes in a number of steps. In each step the removal of the suffix is made to depend upon the form of the remaining stem, which usually involves a measure of its syllable length.

7,572 citations

Proceedings ArticleDOI
08 Feb 1999
TL;DR: Support vector machines for dynamic reconstruction of a chaotic system, Klaus-Robert Muller et al pairwise classification and support vector machines, Ulrich Kressel.
Abstract: Introduction to support vector learning roadmap. Part 1 Theory: three remarks on the support vector method of function estimation, Vladimir Vapnik generalization performance of support vector machines and other pattern classifiers, Peter Bartlett and John Shawe-Taylor Bayesian voting schemes and large margin classifiers, Nello Cristianini and John Shawe-Taylor support vector machines, reproducing kernel Hilbert spaces, and randomized GACV, Grace Wahba geometry and invariance in kernel based methods, Christopher J.C. Burges on the annealed VC entropy for margin classifiers - a statistical mechanics study, Manfred Opper entropy numbers, operators and support vector kernels, Robert C. Williamson et al. Part 2 Implementations: solving the quadratic programming problem arising in support vector classification, Linda Kaufman making large-scale support vector machine learning practical, Thorsten Joachims fast training of support vector machines using sequential minimal optimization, John C. Platt. Part 3 Applications: support vector machines for dynamic reconstruction of a chaotic system, Davide Mattera and Simon Haykin using support vector machines for time series prediction, Klaus-Robert Muller et al pairwise classification and support vector machines, Ulrich Kressel. Part 4 Extensions of the algorithm: reducing the run-time complexity in support vector machines, Edgar E. Osuna and Federico Girosi support vector regression with ANOVA decomposition kernels, Mark O. Stitson et al support vector density estimation, Jason Weston et al combining support vector and mathematical programming methods for classification, Bernhard Scholkopf et al.

5,506 citations


"An Investigation on Linear SVM and ..." refers methods in this paper

  • ...In this paper, we have investigated the effectiveness and suitability of eight linear SVM implementations for TC tasks....

    [...]

  • ...The ohsumed corpus compiled by William hersh [10] consists of medline documents from the year 1981 to 1991....

    [...]

01 Jan 1999
TL;DR: SMO breaks this large quadratic programming problem into a series of smallest possible QP problems, which avoids using a time-consuming numerical QP optimization as an inner loop and hence SMO is fastest for linear SVMs and sparse data sets.

5,350 citations


"An Investigation on Linear SVM and ..." refers methods in this paper

  • ...For SMO algorithm, we used its implementation available in Weka 3.5.4 machine learning workbench [17]....

    [...]

  • ...However, for SMO and ν − SVM algorithms we used posterior probabilities provided by Weka and LIBSVM implementations....

    [...]

  • ...We considered Sequential Minimal Optimization (SMO) [1], SVMlight [2], ν − SVM [3], Smooth SVM (SSVM) [4], Lagrangian SVM (LSVM) [5], Proximal SVM (PSVM) [6], SVMlin [7], and SVMperf [8] for comparison....

    [...]

  • ...We considered Sequential Minimal Optimization (SMO) [1], SVM [2], ν − SVM [3], Smooth SVM (SSVM) [4], Lagrangian SVM (LSVM) [5], Proximal SVM (PSVM) [6], SVMlin [7], and SVM [8] for comparison....

    [...]

  • ...SMO and PSVM algorithms have competitive performance with respect to other algorithms....

    [...]