Showing papers on "Relevance vector machine published in 2011"

LIBSVM: A library for support vector machines

Abstract: LIBSVM is a library for Support Vector Machines (SVMs). We have been actively developing this package since the year 2000. The goal is to help users to easily apply SVM to their applications. LIBSVM has gained wide popularity in machine learning and many other areas. In this article, we present all implementation details of LIBSVM. Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.

Improvements on Twin Support Vector Machines

Abstract: For classification problems, the generalized eigenvalue proximal support vector machine (GEPSVM) and twin support vector machine (TWSVM) are regarded as milestones in the development of the powerful SVMs, as they use the nonparallel hyperplane classifiers. In this brief, we propose an improved version, named twin bounded support vector machines (TBSVM), based on TWSVM. The significant advantage of our TBSVM over TWSVM is that the structural risk minimization principle is implemented by introducing the regularization term. This embodies the marrow of statistical learning theory, so this modification can improve the performance of classification. In addition, the successive overrelaxation technique is used to solve the optimization problems to speed up the training procedure. Experimental results show the effectiveness of our method in both computation time and classification accuracy, and therefore confirm the above conclusion further.

Intelligent prognostics for battery health monitoring based on sample entropy

Abstract: In this paper, an intelligent prognostic for battery health based on sample entropy (SampEn) feature of discharge voltage is proposed. SampEn can provide computational means for assessing the predictability of a time series and also can quantity the regularity of a data sequence. Therefore, when it is applied to discharge voltage battery data, it could serve an indicator for battery health. In this work, the intelligent ability is introduced by utilizing machine learning methods namely support vector machine (SVM) and relevance vector machine (RVM). SampEn and estimated state of charge (SOH) are employed as data input and target vector of learning algorithms, respectively. The results show that the proposed method is plausible due to the good performance of SVM and RVM in SOH prediction. In our study, RVM outperforms SVM based battery health prognostics.

Bayesian Compressive Sampling for Pattern Synthesis With Maximally Sparse Non-Uniform Linear Arrays

Abstract: A numerically-efficient technique based on the Bayesian compressive sampling (BCS) for the design of maximally-sparse linear arrays is introduced. The method is based on a probabilistic formulation of the array synthesis and it exploits a fast relevance vector machine (RVM) for the problem solution. The proposed approach allows the design of linear arrangements fitting desired power patterns with a reduced number of non-uniformly spaced active elements. The numerical validation assesses the effectiveness and computational efficiency of the proposed approach as a suitable complement to existing state-of-the-art techniques for the design of sparse arrays.

Learning with Support Vector Machines

Abstract: Support Vectors Machines have become a well established tool within machine learning. They work well in practice and have now been used across a wide range of applications from recognizing hand-written digits, to face identification, text categorisation, bioinformatics, and database marketing. In this book we give an introductory overview of this subject. We start with a simple Support Vector Machine for performing binary classification before considering multi-class classification and learning in the presence of noise. We show that this framework can be extended to many other scenarios such as prediction with real-valued outputs, novelty detection and the handling of complex output structures such as parse trees. Finally, we give an overview of the main types of kernels which are used in practice and how to learn and make predictions from multiple types of input data. Table of Contents: Support Vector Machines for Classification / Kernel-based Models / Learning with Kernels

A Bayesian-Compressive-Sampling-Based Inversion for Imaging Sparse Scatterers

Abstract: In this paper, a new approach based on the Bayesian compressive sampling (BCS ) and within the contrast source formulation of an inverse scattering problem is proposed for imaging sparse scatterers. By enforcing a probabilistic hierarchical prior as a sparsity regularization constraint, the problem is solved by means of a fast relevance vector machine. The effectiveness and robustness of the BCS-based approach are assessed through a set of numerical experiments concerned with various scatterer configurations and different noisy conditions.

Multi component fault diagnosis of rotational mechanical system based on decision tree and support vector machine

Abstract: The shaft and bearing are the most critical components in rotating machinery. Majority of problems arise from faulty bearings in turn affect the shaft. The vibration signals are widely used to determine the condition of machine elements. The vibration signals are used to extract the features to identify the status of a machine. This paper presents the use of c-SVC and nu-SVC models of support vector machine (SVM) with four kernel functions for classification of faults using statistical features extracted from vibration signals under good and faulty conditions of rotational mechanical system. Decision tree algorithm was used to select the prominent features. These features were given as inputs for training and testing the c-SVC and nu-SVC model of SVM and their fault classification accuracies were compared.

Novelty detection for time series data analysis in water distribution systems using support vector machines

Abstract: The sampling frequency and quantity of time series data collected from water distribution systems has been increasing in recent years, giving rise to the potential for improving system knowledge if suitable automated techniques can be applied, in particular, machine learning. Novelty (or anomaly) detection refers to the automatic identification of novel or abnormal patterns embedded in large amounts of “normal” data. When dealing with time series data (transformed into vectors), this means abnormal events embedded amongst many normal time series points. The support vector machine is a data-driven statistical technique that has been developed as a tool for classification and regression. The key features include statistical robustness with respect to non-Gaussian errors and outliers, the selection of the decision boundary in a principled way, and the introduction of nonlinearity in the feature space without explicitly requiring a nonlinear algorithm by means of kernel functions. In this research, support vector regression is used as a learning method for anomaly detection from water flow and pressure time series data. No use is made of past event histories collected through other information sources. The support vector regression methodology, whose robustness derives from the training error function, is applied to a case study.

An Overview on Theory and Algorithm of Support Vector Machines

Abstract: Statistical learning theory is the statistical theory of smallsample,and it focuses on the statistical law and the nature of learning of small samples.Support vector machine is a new machine learning method based on statistical learning theory,and it has become the research field of machine learning because of its excellent performance.This paper describes the theoretical basis of support vector machines(SVM) systematically,sums up the mainstream machine training algorithms of traditional SVM and some new learning models and algorithms detailedly,and finally points out the research and development prospects of support vector machine.

A hybrid model of self-organizing maps (SOM) and least square support vector machine (LSSVM) for time-series forecasting

Abstract: Support vector machine is a new tool from Artificial Intelligence (AI) field has been successfully applied for a wide variety of problem especially in time-series forecasting. In this paper, least square support vector machine (LSSVM) is an improved algorithm based on SVM, with the combination of self-organizing maps(SOM) also known as SOM-LSSVM is proposed for time-series forecasting. The objective of this paper is to examine the flexibility of SOM-LSSVM by comparing it with a single LSSVM model. To assess the effectiveness of SOM-LSSVM model, two well-known datasets known as the Wolf yearly sunspot data and the Monthly unemployed young women data are used in this study. The experiment shows SOM-LSSVM outperforms the single LSSVM model based on the criteria of mean absolute error (MAE) and root mean square error (RMSE). It also indicates that SOM-LSSVM provides a promising alternative technique in time-series forecasting.

Robust Hyperspectral Classification Using Relevance Vector Machine

Abstract: The curse of dimensionality is the main reason for the computational complexity and the Hughes phenomenon in supervised hyperspectral classification. Previous studies seldom consider in a simultaneous fashion the real situation of insufficiency of available training samples, particularly for small land covers that often contain the key information of the scene, and the problem of complexity. In this paper, the capabilities of a feature reduction technique used for discrimination are combined with the advantages of a Bayesian learning-based probabilistic sparse kernel model, the relevance vector machine (RVM), to develop a new supervised classification method. In the proposed method, the hyperdimensional data are first transformed to a lower dimensionality feature space using the feature reduction technique to maximize separability between classes. The transformed data are then processed by a multiclass RVM classifier based on the parallel architecture and one-against-one strategy. To verify the effectiveness of the method, experiments were carried out on real hyperspectral data. The results are compared with the most efficient supervised classification techniques such as the support vector machine using appropriate performance indicators. The results show that the proposed method performs better than the other approaches particularly for small and scattered landcover classes which are harder to be precisely classified. In addition, this method has the advantages of low computational complexity and robustness to the Hughes phenomenon.

Multi-class Support Vector Machine (SVM) Classifiers -- An Application in Hypothyroid Detection and Classification

Abstract: The paper presents a Multi-class Support Vector Machine classifier and its application to hypothyroid detection and classification. Support Vector Machines (SVM) have been well known method in the machine learning community for binary classification problems. Multi-class SVMs (MCSVM) are usually implemented by combining several binary SVMs. The objective of this work is to show: first, robustness of various kind of kernels for Multi-class SVM classifier, second, a comparison of different constructing methods for Multi-class SVM, such as One-Against-One and One-Against-All, and finally comparing the classifiers' accuracy of Multi-class SVM classifier to AdaBoost and Decision Tree. The simulation results show that One-Against-All Support Vector Machines (OAASVM) are superior to One-Against-One Support Vector Machines (OAOSVM) with polynomial kernels. The accuracy of OAASVM is also higher than AdaBoost and Decision Tree classifier on hypothyroid disease datasets from UCI machine learning dataset.

Parameterizing Support Vector Machines for Land Cover Classification

Abstract: The support vector machine is a group of relatively novel statistical learning algorithms that have not been extensively exploited in the remote sensing community. In previous studies they have been found to generally outperform some popular classifiers. Several recent studies found that training samples and input data dimensionalities can affect image classification accuracies by those popular classifiers and support vector machines alike. The current study extends beyond these recent research frameworks and into another important inquiry area addressing the impacts of internal parameterization on the performance of support vector machines for land-cover classification. A set of support vector machines with different combinations of kernel types, parameters, and error penalty are carefully constructed to classify a Landsat Thematic Mapper image into eight major land-cover categories using identical training data. The accuracy of each classified map is further evaluated using identical reference data. The results reveal that kernel types and error penalty can substantially affect the classification accuracy, and that a careful selection of parameter settings can help improve the performance of the support vector classification. These findings reported here can help establish a practical guidance on the use of support vector machines for land-cover classification from remote sensor data.

Application of relevance vector machine and survival probability to machine degradation assessment

Abstract: Condition monitoring (CM) of machines health or industrial components and systems that can detect, classify and predict the impending faults is critical in reducing operating and maintenance cost. Many papers have reported the valuable models and methods of prognostic systems. However, it was rarely found the papers deal with censored data, which was common in machine condition monitoring practice. This work deals with development of machine degradation assessment system that utilizes censored and complete data collected from CM routine. Relevance vector machine (RVM) is selected as intelligent system then trained by input data obtained from run-to-failure bearing data and target vectors of survival probability estimated by Kaplan-Meier (KM) and probability density function estimators. After validation process, RVM is employed to predict survival probability of individual unit of machine component. The plausibility of the proposed method is shown by applying the proposed method to bearing degradation data in predicting survival probability of individual unit.

Face recognition method based on support vector machine and particle swarm optimization

Abstract: Face recognition belongs to the problem of non-linear, which increases the difficulty of its recognition. Support vector machine (SVM) is a novel machine learning method, which can find global optimum solutions for problems with small training samples and non-linear, so support vector machine has a good application prospect in face recognition. In the study, the novel face recognition method based on support vector machine and particle swarm optimization (PSO-SVM) is presented. In PSO-SVM, PSO is used to simultaneously optimize the parameters of SVM. FERET human face database is adopted to study the face recognition performance of PSO-SVM, and the proposed method is compared with SVM, BPNN. The experimental indicates that PSO-SVM has higher face recognition accuracy than normal SVM, BPNN. Therefore, PSO-SVM is well chosen in face recognition.

A new weighted approach to imbalanced data classification problem via support vector machine with quadratic cost function

Abstract: In this paper, a new weighted approach on Lagrangian support vector machine for imbalanced data classification problem is proposed. The weight parameters are embedded in the Lagrangian SVM formulation. The training method for weighted Lagrangian SVM is presented and its convergence is proven. The weighted Lagrangian SVM classifier is tested and compared with some other SVMs using synthetic and real data to show its effectiveness and feasibility.

Fast Variational Sparse Bayesian Learning With Automatic Relevance Determination for Superimposed Signals

Abstract: In this work, a new fast variational sparse Bayesian learning (SBL) approach with automatic relevance determination (ARD) is proposed. The sparse Bayesian modeling, exemplified by the relevance vector machine (RVM), allows a sparse regression or classification function to be constructed as a linear combination of a few basis functions. It is demonstrated that, by computing the stationary points of the variational update expressions with noninformative (ARD) hyperpriors, a fast version of variational SBL can be constructed. Analysis of the computed stationary points indicates that SBL with Gaussian sparsity priors and noninformative hyperpriors corresponds to removing components with signal-to-noise ratio below a 0 dB threshold; this threshold can also be adjusted to significantly improve the convergence rate and sparsity of SBL. It is demonstrated that the pruning conditions derived for fast variational SBL coincide with those obtained for fast marginal likelihood maximization; moreover, the parameters that maximize the variational lower bound also maximize the marginal likelihood function. The effectiveness of fast variational SBL is demonstrated with synthetic as well as with real data.

Combined Probability Approach and Indirect Data-Driven Method for Bearing Degradation Prognostics

Abstract: This study proposes an application of relevance vector machine (RVM), logistic regression (LR), and autoregressive moving average/generalized autoregressive conditional heteroscedasticity (ARMA/GARCH) models to assess failure degradation based on run-to-failure bearing simulating data. Failure degradation is calculated by using an LR model, and then regarded as the target vectors of the failure probability for training the RVM model. A multi-step-ahead method-based ARMA/GARCH is used to predict censored data, and its prediction performance is compared with one of Dempster-Shafer regression (DSR) method. Furthermore, RVM is selected as an intelligent system, and trained by run-to-failure bearing data and the target vectors of failure probability obtained from the LR model. After training, RVM is employed to predict the failure probability of individual units of bearing samples. In addition, statistical process control is used to analyze the variance of the failure probability. The result shows the novelty of the proposed method, which can be considered as a valid machine degradation prognostic model.

Hybrid robust support vector machines for regression with outliers

Abstract: In this study, a hybrid robust support vector machine for regression is proposed to deal with training data sets with outliers. The proposed approach consists of two stages of strategies. The first stage is for data preprocessing and a support vector machine for regression is used to filter out outliers in the training data set. Since the outliers in the training data set are removed, the concept of robust statistic is not needed for reducing the outliers' effects in the later stage. Then, the training data set except for outliers, called as the reduced training data set, is directly used in training the non-robust least squares support vector machines for regression (LS-SVMR) or the non-robust support vector regression networks (SVRNs) in the second stage. Consequently, the learning mechanism of the proposed approach is much easier than that of the robust support vector regression networks (RSVRNs) approach and of the weighted LS-SVMR approach. Based on the simulation results, the performance of the proposed approach with non-robust LS-SVMR is superior to the weighted LS-SVMR approach when the outliers exist. Moreover, the performance of the proposed approach with non-robust SVRNs is also superior to the RSVRNs approach.

Evolutionary model selection in a wavelet-based support vector machine for automated seizure detection

Abstract: Support vector machines (SVM) have in recent years been gainfully used in various pattern recognition applications. Based on statistical learning theory, this paradigm promises strong robustness to noise and generalization to unseen data. As in any classification technique, appropriate choice of the kernels and input features play an important role in SVM performance. In this study, an evolutionary scheme searches for optimal kernel types and parameters for automated seizure detection. We consider the Lyapunov exponent, fractal dimension and wavelet entropy for possible feature extraction. The classification accuracy of this approach is examined by applying the MIT (Massachusetts Institute of Technology) dataset and comparing results with the SVM. The MIT-BIH dataset has the electrocardiographic (ECG) changes in patients with partial epilepsy which two types ECG beats (partial epilepsy and normal). A comparison of results shows that performance of the evolutionary scheme outweighs that of support vector machine. In the best condition, the accuracy rate of the proposed approaches reaches 100% for specificity and 96.29% for sensitivity.

Measure transformer semantics for Bayesian machine learning

Abstract: The Bayesian approach to machine learning amounts to inferring posterior distributions of random variables from a probabilistic model of how the variables are related (that is, a prior distribution) and a set of observations of variables There is a trend in machine learning towards expressing Bayesian models as probabilistic programs As a foundation for this kind of programming, we propose a core functional calculus with primitives for sampling prior distributions and observing variables We define combinators for measure transformers, based on theorems in measure theory, and use these to give a rigorous semantics to our core calculus The original features of our semantics include its support for discrete, continuous, and hybrid measures, and, in particular, for observations of zero-probability events We compile our core language to a small imperative language that has a straightforward semantics via factor graphs, data structures that enable many efficient inference algorithms We use an existing inference engine for efficient approximate inference of posterior marginal distributions, treating thousands of observations per second for large instances of realistic models

Simple estimate of the width in Gaussian kernel with adaptive scaling technique

Abstract: This paper presents a simple method to estimate the width of Gaussian kernel based on an adaptive scaling technique. The Gaussian kernel is widely employed in radial basis function (RBF) network, support vector machine (SVM), least squares support vector machine (LS-SVM), Kriging models, and so on. It is widely known that the width of the Gaussian kernel in these machine learning techniques plays an important role. Determination of the optimal width is a time-consuming task. Therefore, it is preferable to determine the width with a simple manner. In this paper, we first examine a simple estimate of the width proposed by Nakayama et al. Through the examination, four sufficient conditions for the simple estimate of the width are described. Then, a new simple estimate for the width is proposed. In order to obtain the proposed estimate of the width, all dimensions are equally scaled. A simple technique called the adaptive scaling technique is also developed. It is expected that the proposed simple method to estimate the width is applicable to wide range of machine learning techniques employing the Gaussian kernel. Through examples, the validity of the proposed simple method to estimate the width is examined.