Support vector machines (SVM) are a powerful tool to analyze data with a number of predictors approximately equal or larger than the number of observations. However, originally, application of SVM to analyze biomedical data was limited because SVM was not designed to evaluate importance of predictor variables. Creating predictor models based on only the most relevant variables is essential in biomedical research. Currently, substantial work has been done to allow assessment of variable importance in SVM models but this work has focused on SVM implemented with linear kernels. The power of SVM as a prediction model is associated with the flexibility generated by use of non-linear kernels. Moreover, SVM has been extended to model survival outcomes. This paper extends the Recursive Feature Elimination (RFE) algorithm by proposing three approaches to rank variables based on non-linear SVM and SVM for survival analysis. The proposed algorithms allows visualization of each one the RFE iterations, and hence, identification of the most relevant predictors of the response variable. Using simulation studies based on time-to-event outcomes and three real datasets, we evaluate the three methods, based on pseudo-samples and kernel principal component analysis, and compare them with the original SVM-RFE algorithm for non-linear kernels. The three algorithms we proposed performed generally better than the gold standard RFE for non-linear kernels, when comparing the truly most relevant variables with the variable ranks produced by each algorithm in simulation studies. Generally, the RFE-pseudo-samples outperformed the other three methods, even when variables were assumed to be correlated in all tested scenarios. The proposed approaches can be implemented with accuracy to select variables and assess direction and strength of associations in analysis of biomedical data using SVM for categorical or time-to-event responses. Conducting variable selection and interpreting direction and strength of associations between predictors and outcomes with the proposed approaches, particularly with the RFE-pseudo-samples approach can be implemented with accuracy when analyzing biomedical data. These approaches, perform better than the classical RFE of Guyon for realistic scenarios about the structure of biomedical data.

/pdf/svm-rfe-selection-and-visualization-of-the-most-relevant-449srrilt2.pdf

SVM-RFE: selection and visualization of the most relevant features through non-linear kernels

Data uncertainty is an inherent property in various applications due to reasons such as outdated sources or imprecise measurement. When data mining techniques are applied to these data, their uncertainty has to be considered to obtain high quality results. We present UK-means clustering, an algorithm that enhances the K-means algorithm to handle data uncertainty. We apply UK-means to the particular pattern of moving-object uncertainty. Experimental results show that by considering uncertainty, a clustering algorithm can produce more accurate results.

Uncertain data mining : An example in clustering location data

https://ro.uow.edu.au/cgi/viewcontent.cgi?article=4512&context=eispapers1

An intelligent blockchain-based system for safe vaccine supply and supervision

Predicting and controlling crystallinity of molecular materials has applications in a crystal engineering context, as well as process control and formulation in the pharmaceutical industry. Here, we present a machine learning approach to this problem which uses a large input training set which is classified on a single measurable outcome: does a substance have a reasonable probability of forming good quality crystals. While the related problem of crystal structure prediction requires reliable calculation of three dimensional molecular conformations, the method employed here for predicting crystallisation propensity uses only “two dimensional” information consisting of atom types and connectivity. We show that an error rate lower than 10% can be achieved against unseen test data. The predictive model was also tested in a blind screen of a set of compounds which do not have crystal structures reported in the literature, and we found it to have a 79% classification accuracy. Analysis of the most significant descriptors used in the classification shows that the number of rotatable bonds and a molecular connectivity index are key in determining crystallisation propensity and using these two measures alone can give 80% accurate classification of unseen test data.

Will it crystallise? Predicting crystallinity of molecular materials

Background: Even though the classification of cancer tissue samples based on gene expression data has advanced considerably in recent years, it faces great challenges to improve accuracy. One of the challenges is to establish an effective method that can select a parsimonious set of relevant genes. So far, most methods for gene selection in literature focus on screening individual or pairs of genes without considering the possible interactions among genes. Here we introduce a new computational method named the Binary Matrix Shuffling Filter (BMSF). It not only overcomes the difficulty associated with the search schemes of traditional wrapper methods and overfitting problem in large dimensional search space but also takes potential gene interactions into account during gene selection. This method, coupled with Support Vector Machine (SVM) for implementation, often selects very small number of genes for easy model interpretability. Results: We applied our method to 9 two-class gene expression datasets involving human cancers. During the gene selection process, the set of genes to be kept in the model was recursively refined and repeatedly updated according to the effect of a given gene on the contributions of other genes in reference to their usefulness in cancer classification. The small number of informative genes selected from each dataset leads to significantly improved leave-one-out (LOOCV) classification accuracy across all 9 datasets for multiple classifiers. Our method also exhibits broad generalization in the genes selected since multiple commonly used classifiers achieved either equivalent or much higher LOOCV accuracy than those reported in literature. Conclusions: Evaluation of a gene’s contribution to binary cancer classification is better to be considered after adjusting for the joint effect of a large number of other genes. A computationally efficient search scheme was provided to perform effective search in the extensive feature space that includes possible interactions of many genes. Performance of the algorithm applied to 9 datasets suggests that it is possible to improve the accuracy of cancer classification by a big margin when joint effects of many genes are considered.

/pdf/improving-accuracy-for-cancer-classification-with-a-new-46lh5xfulb.pdf

Improving accuracy for cancer classification with a new algorithm for genes selection

Linear kernel Support Vector Machine Recursive Feature Elimination (SVM-RFE) is known as an excellent feature selection algorithm. Nonlinear SVM is a black box classifier for which we do not know the mapping function $${\Phi}$$ explicitly. Thus, the weight vector w cannot be explicitly computed. In this paper, we proposed a feature selection algorithm utilizing Support Vector Machine with RBF kernel based on Recursive Feature Elimination(SVM-RBF-RFE), which expands nonlinear RBF kernel into its Maclaurin series, and then the weight vector w is computed from the series according to the contribution made to classification hyperplane by each feature. Using $${w_i^2}$$ as ranking criterion, SVM-RBF-RFE starts with all the features, and eliminates one feature with the least squared weight at each step until all the features are ranked. We use SVM and KNN classifiers to evaluate nested subsets of features selected by SVM-RBF-RFE. Experimental results based on 3 UCI and 3 microarray datasets show SVM-RBF-RFE generally performs better than information gain and SVM-RFE.

Feature selection for support vector machines with RBF kernel

Uncertain data is ubiquitous in real-world applications due to various causes. In recent years, clustering uncertain data has been paid more attention by the research community, and the classical clustering algorithms based on partition, density and hierarchy have been extended to handle the uncertain data. However, these extended algorithms usually work in the input space. In this paper, to well explore the inherent data pattern in the high dimensional feature space, we propose a kernel based K-medoids algorithm for clustering uncertain data. Extensive experiments performed on synthetic and several real datasets demonstrate that our kernel based method has higher clustering accuracy than the state-of the - art UK-medoids algorithm. Also, it signifies that the uncertain data pattern in the new feature space could be well presented when the kernel function and the K-medoids algorithm are effectively incorporated.

Kernel based K-medoids for clustering data with uncertainty

Uncertainty exists widely in real-word applications. Recently, the research for uncertain data has attracted more and more attention. While not enough attention has been paid to the research of cost- sensitive algorithm on uncertain data. In this paper, we propose a simple but effective method to extend traditional cost-sensitive decision tree to uncertain data, and the algorithm can deal with both certain and uncertain data. In our experiment, we compare the proposed algorithm with DTU[18] on UCI datasets. The experimental result proves that the proposed algorithm performs better than DTU, with lower computational complexity. It keeps low cost even at high level of uncertainty, which makes it applicable to real-life applications for data uncertainty.

Cost-sensitive decision tree for uncertain data

Traditional approach for building text classifiers usually require a lot of labeled documents, which are expensive to obtain. In this paper, we propose a new text classification approach based on a keyword and Wikipedia knowledge, so as to avoid labeling documents manually. Firstly, we retrieve a set of related documents about the keyword from Wikipedia. And then, with the help of related Wikipedia pages, more positive documents are extracted from the unlabeled documents. Finally, we train a text classifier with these positive documents and unlabeled documents. The experiment result on 20Newsgroup dataset show that the proposed approach performs very competitively compared with NB-SVM, a PU learner, and NB, a supervised learner.

Building a Text Classifier by a Keyword and Wikipedia Knowledge

Sigmoid kernel is widely applied in neural networks for classification tasks. SVM classifier, which is applied with sigmoid kernel, has excellent classification accuracy. However, as sigmoid kernel has complicated structure, it is generally difficult for human expert to interpret and understand how the sigmoid kernel makes its classification decision. As decision rule classifier is understandable to human expert, in this paper, we present our InterSIG algorithm, which mines decision rules from the classification hyper-plane which is constructed by SVM with sigmoid kernel. InterSIG expands sigmoid kernel into its Maclaurin series, and then mines classification rules which make great contribution to classification from the classification hyper-plane. Experiment results show that InterSIG classifier is more understandable to human experts without jeopardizing the accuracy than the original SVM with sigmoid kernel. Furthermore, compared with 3 association classifiers, CMAR, CBA, CPAR and C4.5, a decision tree classifier, InterSIG classifier is very encouraging over the 9 datasets.

Yang Zhang

Papers

Feature selection for support vector machines with RBF kernel

Kernel based K-medoids for clustering data with uncertainty

Cost-sensitive decision tree for uncertain data

Building a Text Classifier by a Keyword and Wikipedia Knowledge

Extracting Decision Rules from Sigmoid Kernel