Showing papers on "Sequential minimal optimization published in 2014"

Multiple Kernel Learning for Visual Object Recognition: A Review

Abstract: Multiple kernel learning (MKL) is a principled approach for selecting and combining kernels for a given recognition task. A number of studies have shown that MKL is a useful tool for object recognition, where each image is represented by multiple sets of features and MKL is applied to combine different feature sets. We review the state-of-the-art for MKL, including different formulations and algorithms for solving the related optimization problems, with the focus on their applications to object recognition. One dilemma faced by practitioners interested in using MKL for object recognition is that different studies often provide conflicting results about the effectiveness and efficiency of MKL. To resolve this, we conduct extensive experiments on standard datasets to evaluate various approaches to MKL for object recognition. We argue that the seemingly contradictory conclusions offered by studies are due to different experimental setups. The conclusions of our study are: (i) given a sufficient number of training examples and feature/kernel types, MKL is more effective for object recognition than simple kernel combination (e.g., choosing the best performing kernel or average of kernels); and (ii) among the various approaches proposed for MKL, the sequential minimal optimization, semi-infinite programming, and level method based ones are computationally most efficient.

Evaluation of different feature sets for gait recognition using skeletal data from Kinect

Abstract: Gait is a persons manner of walking. It is a biometric that can be used for identifying humans. Gait is an unobtrusive metric that can be obtained from distance, and this is its main strength compared to other biometrics. In this paper we construct and evaluate feature sets with the purpose of finding out the role of different types of features and body parts in the recognition process. The feature sets were constructed from skeletal images in three dimensions made with a Kinect sensor. The Kinect is a low-cost device that includes RGB, depth and audio sensors. In our work automated gait cycle extraction algorithm was performed on the Kinect recordings. Metrics like angles and distances between joints were aggregated within a gait cycle, and from those aggregations the different feature datasets were constructed. Multilayer perceptron, support vector machine with sequential minimal optimization and J48 algorithms were used for classification on these datasets. At the end we give conclusions on which groups of features and body parts gave the best recognition rates.

Training SVMs on a bound vectors set based on Fisher projection

Abstract: Standard support vector machines (SVMs) training algorithms have O(l 3) computational and O(l 2) space complexities, where l is the training set size. It is thus computationally infeasible on very large data sets. To alleviate the computational burden in SVM training, we propose an algorithm to train SVMs on a bound vectors set that is extracted based on Fisher projection. For linear separate problems, we use linear Fisher discriminant to compute the projection line, while for non-linear separate problems, we use kernel Fisher discriminant to compute the projection line. For each case, we select a certain ratio samples whose projections are adjacent to those of the other class as bound vectors. Theoretical analysis shows that the proposed algorithm is with low computational and space complexities. Extensive experiments on several classification benchmarks demonstrate the effectiveness of our approach.

CFS–SMO based classification of breast density using multiple texture models

Abstract: It is highly acknowledged in the medical profession that density of breast tissue is a major cause for the growth of breast cancer. Increased breast density was found to be linked with an increased risk of breast cancer growth, as high density makes it difficult for radiologists to see an abnormality which leads to false negative results. Therefore, there is need for the development of highly efficient techniques for breast tissue classification based on density. This paper presents a hybrid scheme for classification of fatty and dense mammograms using correlation-based feature selection (CFS) and sequential minimal optimization (SMO). In this work, texture analysis is done on a region of interest selected from the mammogram. Various texture models have been used to quantify the texture of parenchymal patterns of breast. To reduce the dimensionality and to identify the features which differentiate between breast tissue densities, CFS is used. Finally, classification is performed using SMO. The performance is evaluated using 322 images of mini-MIAS database. Highest accuracy of 96.46 % is obtained for two-class problem (fatty and dense) using proposed approach. Performance of selected features by CFS is also evaluated by Naive Bayes, Multilayer Perceptron, RBF Network, J48 and kNN classifier. The proposed CFS–SMO method outperforms all other classifiers giving a sensitivity of 100 %. This makes it suitable to be taken as a second opinion in classifying breast tissue density.

Automated Microalgae Image Classification

Abstract: In this paper we present a new method for automated recognition of 12 microalgae that are most commonly found in water resources of Thailand. In order to handle some difficulties encountered in our problem such as unclear algae boundary and noisy background, we proposed a new method for segmenting algae bodies from an image background and proposed a new method for computing texture descriptors from a blurry texture object. Feature combination approach is applied to handle a variation of algae shapes of the same genus. Sequential Minimal Optimization (SMO) is used as a classifier. An experimental result of 97.22% classification accuracy demonstrates an effectiveness of our proposed method.

A Unifying Framework in Vector-valued Reproducing Kernel Hilbert Spaces for Manifold Regularization and Co-Regularized Multi-view Learning

Abstract: This paper presents a general vector-valued reproducing kernel Hilbert spaces (RKHS) framework for the problem of learning an unknown functional dependency between a structured input space and a structured output space. Our formulation encompasses both Vector-valued Manifold Regularization and Co-regularized Multi-view Learning, providing in particular a unifying framework linking these two important learning approaches. In the case of the least square loss function, we provide a closed form solution, which is obtained by solving a system of linear equations. In the case of Support Vector Machine (SVM) classification, our formulation generalizes in particular both the binary Laplacian SVM to the multi-class, multi-view settings and the multi-class Simplex Cone SVM to the semi-supervised, multi-view settings. The solution is obtained by solving a single quadratic optimization problem, as in standard SVM, via the Sequential Minimal Optimization (SMO) approach. Empirical results obtained on the task of object recognition, using several challenging datasets, demonstrate the competitiveness of our algorithms compared with other state-of-the-art methods.

Identification of biomarkers that distinguish chemical contaminants based on gene expression profiles

Abstract: High throughput transcriptomics profiles such as those generated using microarrays have been useful in identifying biomarkers for different classification and toxicity prediction purposes. Here, we investigated the use of microarrays to predict chemical toxicants and their possible mechanisms of action. In this study, in vitro cultures of primary rat hepatocytes were exposed to 105 chemicals and vehicle controls, representing 14 compound classes. We comprehensively compared various normalization of gene expression profiles, feature selection and classification algorithms for the classification of these 105 chemicals into14 compound classes. We found that normalization had little effect on the averaged classification accuracy. Two support vector machine (SVM) methods, LibSVM and sequential minimal optimization, had better classification performance than other methods. SVM recursive feature selection (SVM-RFE) had the highest overfitting rate when an independent dataset was used for a prediction. Therefore, we developed a new feature selection algorithm called gradient method that had a relatively high training classification as well as prediction accuracy with the lowest overfitting rate of the methods tested. Analysis of biomarkers that distinguished the 14 classes of compounds identified a group of genes principally involved in cell cycle function that were significantly downregulated by metal and inflammatory compounds, but were induced by anti-microbial, cancer related drugs, pesticides, and PXR mediators. Our results indicate that using microarrays and a supervised machine learning approach to predict chemical toxicants, their potential toxicity and mechanisms of action is practical and efficient. Choosing the right feature and classification algorithms for this multiple category classification and prediction is critical.

Parallel Support Vector Machines in Practice

Abstract: In this paper, we evaluate the performance of various parallel optimization methods for Kernel Support Vector Machines on multicore CPUs and GPUs. In particular, we provide the first comparison of algorithms with explicit and implicit parallelization. Most existing parallel implementations for multi-core or GPU architectures are based on explicit parallelization of Sequential Minimal Optimization (SMO)---the programmers identified parallelizable components and hand-parallelized them, specifically tuned for a particular architecture. We compare these approaches with each other and with implicitly parallelized algorithms---where the algorithm is expressed such that most of the work is done within few iterations with large dense linear algebra operations. These can be computed with highly-optimized libraries, that are carefully parallelized for a large variety of parallel platforms. We highlight the advantages and disadvantages of both approaches and compare them on various benchmark data sets. We find an approximate implicitly parallel algorithm which is surprisingly efficient, permits a much simpler implementation, and leads to unprecedented speedups in SVM training.

Efficient source mask optimization with Zernike polynomial functions for source representation

Abstract: In 22nm optical lithography and beyond, source mask optimization (SMO) becomes vital for the continuation of advanced ArF technology node development. The pixel-based method permits a large solution space, but involves a time-consuming optimization procedure because of the large number of pixel variables. In this paper, we introduce the Zernike polynomials as basis functions to represent the source patterns, and propose an improved SMO algorithm with this representation. The source patterns are decomposed into the weighted superposition of some well-chosen Zernike polynomial functions, and the number of variables decreases significantly. We compare the computation efficiency and optimization performance between the proposed method and the conventional pixel-based algorithm. Simulation results demonstrate that the former can obtain substantial speedup of source optimization while improving the pattern fidelity at the same time.

Comparing results of thermographic images based diagnosis for breast diseases

Abstract: This paper examines the potential contribution of infrared (IR) imaging in breast diseases detection. It compares obtained results using some algorithms for detection of malignant breast conditions such as Support Vector Machine (SVM) regarding the consistency of different approaches when applied to public data. Moreover, in order to avail the actual IR imaging's capability as a complement on clinical trials and to promote researches using high-resolution IR imaging we deemed the use of a public database revised by confidently trained breast physicians as essential. Only the static acquisition protocol is regarded in our work. We used 102 IR single breast images from the Pro Engenharia (PROENG) public database (54 normal and 48 with some finding). These images were collected from Universidade Federal de Pernambuco (UFPE) University?s Hospital. We employed the same features proposed by the authors of the work that presented the best results and achieved an accuracy of 61.7 % and Youden index of 0.24 using the Sequential Minimal Optimization (SMO) classifier.

Combination of One-Class Support Vector Machines for Classification with Reject Option

Abstract: This paper focuses on binary classification with reject option, enabling the classifier to detect and abstain hazardous decisions. While reject classification produces in more reliable decisions, there is a tradeoff between accuracy and rejection rate. Two type of rejection are considered: ambiguity and outlier rejection. The state of the art mostly handles ambiguity rejection and ignored outlier rejection. The proposed approach, referred as CONSUM, handles both ambiguity and outliers detection. Our method is based on a quadratic constrained optimization formulation, combining one-class support vector machines. An adaptation of the sequential minimal optimization algorithm is proposed to solve the minimization problem. The experimental study on both artificial and real world datasets exams the sensitivity of the CONSUM with respect to the hyper-parameters and demonstrates the superiority of our approach.

Text classification method based on chi square statistics and SMO algorithm

Abstract: The invention discloses a text classification method based on chi square statistics and an SMO algorithm. The method comprises the steps that first, training texts are subjected to word segmentation, stop word removing and preprocessing, and then a chi square statistics quantity is used as a standard for selecting a set number of words to be used as feature words; then, the feature weight values of the training texts and testing texts are computed respectively; feature vectors of each training text and each testing text are converted into training document vector models and testing document vector models; and a trained classifier carries out classification on the feature vectors of the testing texts, and the classifying result of each testing text is obtained. According the method, the shortcomings that a lot of text classification features and a lot of noise exist due to the fact that all words are used as features can be overcome, and text classification accuracy and efficiency can be improved.

Chinese Question Classification Based on Question Property Kernel

Abstract: Support vector machine have been widely used in classification tasks, however, the structure of the question is ignored while using the standard kernel function in the question classification. To solve the problem, a question property kernel function which combines syntactic dependency relationship and POS (part of speech) is proposed in this paper. Firstly we extract the term, POS, dependency relationship of "HED" words and dependency relationship of "question words" from questions. And then we adopt the value of kernel function by computing the dependency relationship of the term, POS, and the dependency path which the two terms shared. At last we get the support vectors by SMO algorithm. The results of experiments show that the kernel function proposed in this paper which implicated the effective utilization of the question structure can improves the accuracy of the classification.

Application of ranking based attribute selection filters to perform automated evaluation of descriptive answers through sequential minimal optimization models

Abstract: In this paper, we study the performance of various models for automated evaluation of descriptive answers by using rank based feature selection filters for dimensionality reduction. We quantitatively analyze the best from amongst five rank based feature selection techniques, namely Chi Squared filter, Information Gain filter, Gain Ratio filter, Relief filter and Symmetrical Uncertainty filter. We use Sequential Minimal Optimization with Polynomial kernel to build models and we evaluate these models in terms of various parameters such as Accuracy, Time to build the models, Kappa, Mean Absolute Error and Root Mean Squared Error. For all except the Relief filter, the accuracies obtained are at least 4% better than the accuracies obtained with the same models without any filters applied. We found that the accuracies recorded are same for Chi Squared filter, Information Gain filter, Gain Ratio filter and Symmetrical Uncertainty filter. Therefore accuracy alone is not the sole determinant in selecting the best filter. The time taken to build the models, Kappa, Mean Absolute Error and Root Mean Squared Error play a major role in determining the effectiveness of these filters. The overall rank aggregation metric of Symmetrical Uncertainty filter is 45 and this is better by 1 rank than the rank aggregation metric of Information gain filter. Symmetric Uncertainty filter’s rank aggregation metric is better by 3, 6, 112 ranks respectively when compared to the rank aggregation metrics of Chi Squared filter, Gain Ratio filter and Relief filters. Through these quantitative measurements, we conclude that Symmetrical Uncertainty attribute evaluation is the overall best performing rank based feature selection algorithm applicable for auto evaluation of descriptive answers.

Music Data Processing and Mining in Large Databases for Active Media

Abstract: The aim of this paper was to investigate the problem of music data processing and mining in large databases. Tests were performed on a large database that included approximately 30000 audio files divided into 11 classes corresponding to music genres with different cardinalities. Every audio file was described by a 173-element feature vector. To reduce the dimensionality of data the Principal Component Analysis (PCA) with variable value of factors was employed. The tests were conducted in the WEKA application with the use of k-Nearest Neighbors (kNN), Bayesian Network (Net) and Sequential Minimal Optimization (SMO) algorithms. All results were analyzed in terms of the recognition rate and computation time efficiency.

Statistical analysis of big data: an approach based on support vector machines for classification and regression problems

Abstract: A new type of learning algorithms with the supervisor for estimating multidimensional functions is considered. These methods based on Support Vector Machines are widely used due to their ability to deal with high-dimensional and large datasets, and their flexibility in modeling diverse sources of data. Support vector machines and related kernel methods are extremely good at solving prediction problems in computational biology. A background about statistical learning theory and kernel feature spaces is given including practical and algorithmic considerations.

Weighted hyper-sphere support vector machine algorithm based image classification method

Abstract: The invention discloses a weighted hyper-sphere support vector machine algorithm based image classification method which can improve the calculation speed and the calculation accuracy The weighted hyper-sphere support vector machine algorithm based image classification method comprises collecting image data; denoising and performing normalization processing on images, calculating characteristics of HOG (Histograms of Oriented Gradients) of the images, adding predefined categories into image categories and classifying the images; calculating a data center of every category, calculating the weight of every sample of the category of data according to the data center and ordering the training samples according to the weight; training a hyper-sphere support vector machine through a multithreading genetic algorithm and an SMO (Sequential Minimal Optimization) method and solving an optimal parameter and a corresponding hyper-sphere model; extracting HOG characteristics of the newly collected images, calculating position relationships between newly collected samples and the hyper-sphere model, obtaining label categories according to a distinguish rule and labelling categories of the newly collected images

A social-spider optimization approach for support vector machines parameters tuning

Abstract: The choice of hyper-parameters in Support Vector Machines (SVM)-based learning is a crucial task, since different values may degrade its performance, as well as can increase the computational burden. In this paper, we introduce a recently developed nature-inspired optimization algorithm to find out suitable values for SVM kernel mapping named Social-Spider Optimization (SSO). We compare the results obtained by SSO against with a Grid-Search, Particle Swarm Optimization and Harmonic Search. Statistical evaluation has showed SSO can outperform the compared techniques for some sort of kernels and datasets.

Incremental Learning with Support Vector Data Description

Abstract: Due to the simplicity and firm mathematical foundation, Support Vector Machines (SVMs) have been intensively used to solve classification problems. However, training SVMs on real world large-scale databases is computationally costly and sometimes infeasible when the dataset size is massive and non-stationary. In this paper, we propose an incremental learning approach that greatly reduces the time consumption and memory usage for training SVMs. The proposed method is fully dynamic, which stores only a small fraction of previous training examples whereas the rest can be discarded. It can further handle unseen labels in new training batches. The classification experiments show that the proposed method achieves the same level of classification accuracy as batch learning while the computational cost is significantly reduced, and it can outperform other incremental SVM approaches for the new class problem.

Exterior-Point Method for Support Vector Machines

Abstract: We present an exterior-point method (EPM) for training dual-soft margin support vector machines (SVMs). The EPM stems from nonlinear rescaling and augmented Lagrangian methods and allows iterates to approach the solution of a constrained nonlinear optimization problem from the exterior of the feasible set. Furthermore, the EPM produces and solves a well-conditioned system of linear equations at each iteration; thus, avoiding numerical inaccuracies that can occur when solving ill-conditioned systems. Therefore, the EPM may be an attractive alternative to existing quadratic programming solvers for training SVMs. We report numerical results for training the SVM with the EPM on data up to several thousand data points from the UC Irvine Machine Learning Repository.

Research on Classification of Hyperspectral Remote Sensing Imagery Based on BDT-SMO and Combined Features

Abstract: There are some prevalent problems in the classification of hyperspectral remote sensing imagery currently, such as many bands, large amount of data, high proportion of mixed pixels and lower spatial resolution and so no. In order to solve the above problems, the sequential minimal optimization (SMO) algorithm is researched, and a supervised classification method based on binary decision tree SMO (BDT-SMO) algorithm and spectrum-texture combined features is proposed to improve the accuracy and efficiency of hyperspectral remote sensing imagery classification. The higher spatial resolution imagery (ALI) and hyperspectral imagery (Hyperion) which have been acquired from the same sensor (EO-1) in the same time are used as the experimental data and implemented geometric correction and pixel-level fusion. Extract the spectral features and textural features of the ground objects from the fusion images, combine the features above and train the BDT-SMO multi-class classifier based on separation degree. The classifier is used for the land-use status classification of experimental areas. Select two different sets of samples which are based on spectral features and spectrum-texture combined features, and use the four different methods of maximum likelihood, BP neural network, BDT-SVM and BDT-SMO to train the two different sets of samples above separately. Experimental results show that the BDT-SMO classification method based on combined features can improve the efficiency and accuracy of land-use status classification effectively and has better generalization ability.

Parallel SMO algorithm implementation based on OpenMP

Abstract: Sequential minimal optimization (SMO) algorithm is widely used for solving the optimization problem during the training process of support vector machine (SVM). However, the SMO algorithm is quite time-consuming when handling very large training sets and thus limits the performance of SVM. In this paper, a parallel implementation of SMO algorithm is designed with OpenMP, basing on the running time analysis of each function in SMO. Experimental results show that the performance for training SVM had been improved with parallel SMO when dealing with large datasets.

Klasifikasi wilayah desa-perdesaan dan desa-perkotaan wilayah kabupaten semarang dengan support vector machine (svm)

Abstract: This research will be carry out classification based on the status of the rural and urban regions that reflect the differences in characteristics/ conditions between regions in Indonesia with Support Vector Machine (SVM) method. Classification on this issue is working by build separation functions involving the kernel function to map the input data into a higher dimensional space. Sequential Minimal Optimization (SMO) algorithms is used in the training process of data classification of rural and urban regions to get the optimal separation function (hyperplane). To determine the kernel function and parameters according to the data, grid search method combined with the leave-one-out cross-validation method is used. In the classification using SVM, accuracy is obtained, which the best value is 90% using Radial Basis Function (RBF) kernel functions with parameters C=100 dan γ=2 -5 . Keywords : classification, support vector machine, sequential minimal optimization, grid search, leave-one-out, cross validation, rural, urban

An Image Retrieval Method Based on Machine Learning and SVM

Abstract: Image recognition has been a research hotspot in the field of machine learning; this paper puts forward a kind of cascade algorithm based on SVM and AdaBoost. The algorithm to select the sample pretreatment, fixed size of window image segmentation into different areas, then using Haar - like rectangular figure characteristics of integral method for feature extraction, finally using AdaBoost cascade classifier to classify the SVM training. Through the face recognition experiments show AdaBoost cascade of SVM algorithm improve the classification accuracy, error rate get reduced obviously.

A Survey on Brain Tumor Detection and Classification System based on Artificial Neural Network

Abstract: Brain Tumour is one of the serious problems among various other existing life threatening diseases. Tumour detection is done initially by MRI , BIOPSY , SPINAL TAPE TEST ,ANNINOGARM and by some other similar kind of tests. All these tests are not only painful but are expensive too. Hence a brain tumour detection and classification system is required for early detection and categorization of tumour . In this paper we will study and analyze already proposed systems and will try to find the efficient and effective approaches. Tumour has a variant and complex structure and hence its classification is difficult .In the first phase Image pre-processing is performed initially on MR images of the patients to enhance features of brain cells and then a neural based classifier is implemented. BPNN, Radial basis and SMO based classifiers are examined. SMO when used with k- means clustering provides a more accurate system. We have a learning phase where ANN is trained or learned by providing some images which are already classified as cancerous and non cancerous. After learning phase classification system is tested by giving some new inputs and comparing the results. Keywordsresonance Imaging (MRI), Back propagation network (BPNN), Sequential minimal optimization (SMO)

Time Series Data Mining in Real Time Surface Runoff Forecasting through Support Vector Machine

Abstract: This study presents support vector machine based model for forecasting the runoff-rainfall events. A SVM based model is either implemented through Radial base or Gaussian based Kernel functions. SVM uses precipitation, temperature, sediment, rainfall, water level and discharge as input variable parameters. In this research the Sequential minimal optimization algorithm (SMO) has been implemented as an effective method for training support vector machines (SVMs) on classification tasks defined on large and sparse real time data sets. In this work, we generalized the SMO so that it can handle regression problem and by dividing datasets into test data and trained data performed future forecasting keeping four major evaluation parameters Root Mean Square Error (RMSE), Mean Absolute error (MAE), Mean Squared error (MSE) and correlation coefficient (CC). Study site for this research is Narmada basin reservoir hosahangabad catchment area and the experimental results on predicting the full natural flow of Narmada River indicates that support vector machine method performs far better and more accurate from the current forecasting practices (Artificial Neural Network).

Text classification based on SMO and fuzzy model

Abstract: In this article we propose a text classification system using chi-value as feature selection method and SMO (sequential minimal optimization) algorithm as classifier. In addition, we use fuzzy model of fuzzy concept to describe documents' classified label and entropy to calculate the uncertainty of a document's classification result. Experimental results demonstrated that the proposed method can reach 87% or higher accuracy of text classification.

Exact SVM training by Wolfe's minimum norm point algorithm

Abstract: This paper applies Wolfe's algorithm for finding the minimum norm point in a polytope to training of standard SVM with hinge loss. The resulting algorithm is guaranteed to obtain an exact optimal solution within a finite number of iterations. Experiments illustrate that our algorithm runs faster than existing algorithms such as LIBSVM for the same model. In comparison with LIBLINEAR, which adopts a variant of SVMs, our approach works better when the feature size is modest; the feature size is sufficiently smaller than the sample size.