Showing papers on "Multiple kernel learning published in 2020"

Identification of Drug–Target Interactions via Dual Laplacian Regularized Least Squares with Multiple Kernel Fusion

Abstract: Detection of Drug–Target Interactions (DTIs) is the time-consuming and laborious experiment via biochemical approaches. Machine learning based methods have been widely used to mine meaningful information of drug research. In this study, we establish a novel computational method to predict DTIs via Dual Laplacian Regularized Least Squares model (DLapRLS) with Hilbert–Schmidt Independence Criterion-based Multiple Kernel Learning (HSIC-MKL). Multiple kernels are built from different information sources (drug and target spaces). Then, above corresponding kernels are integrated by HSIC-MKL. At last, DLapRLS model is trained by Alternating Least Squares Algorithm (ALSA) and employed to predict new DTIs. On four benchmark datasets, the results of our method are comparable and even better than existing models.

Multiple Kernel Learning-Based Transfer Regression for Electric Load Forecasting

Abstract: Electric load forecasting, especially short-term load forecasting (STLF), is becoming more and more important for power system operation. We propose to use multiple kernel learning (MKL) for residential electric load forecasting which provides more flexibility than traditional kernel methods. Computation time is an important issue for short-term forecasting, especially for energy scheduling. However, conventional MKL methods usually lead to complicated optimization problems. Another practical issue for this application is that there may be a very limited amount of data available to train a reliable forecasting model for a new house, while at the same time we may have historical data collected from other houses which can be leveraged to improve the prediction performance for the new house. In this paper, we propose a boosting-based framework for MKL regression to deal with the aforementioned issues for STLF. In particular, we first adopt boosting to learn an ensemble of multiple kernel regressors and then extend this framework to the context of transfer learning. Furthermore, we consider two different settings: homogeneous transfer learning and heterogeneous transfer learning. Experimental results on residential data sets demonstrate that forecasting error can be reduced by a large margin with the knowledge learned from other houses.

Multiple Kernel Clustering With Neighbor-Kernel Subspace Segmentation

Abstract: Multiple kernel clustering (MKC) has been intensively studied during the last few decades. Even though they demonstrate promising clustering performance in various applications, existing MKC algorithms do not sufficiently consider the intrinsic neighborhood structure among base kernels, which could adversely affect the clustering performance. In this paper, we propose a simple yet effective neighbor-kernel-based MKC algorithm to address this issue. Specifically, we first define a neighbor kernel, which can be utilized to preserve the block diagonal structure and strengthen the robustness against noise and outliers among base kernels. After that, we linearly combine these base neighbor kernels to extract a consensus affinity matrix through an exact-rank-constrained subspace segmentation. The naturally possessed block diagonal structure of neighbor kernels better serves the subsequent subspace segmentation, and in turn, the extracted shared structure is further refined through subspace segmentation based on the combined neighbor kernels. In this manner, the above two learning processes can be seamlessly coupled and negotiate with each other to achieve better clustering. Furthermore, we carefully design an efficient iterative optimization algorithm with proven convergence to address the resultant optimization problem. As a by-product, we reveal an interesting insight into the exact-rank constraint in ridge regression by careful theoretical analysis: it back-projects the solution of the unconstrained counterpart to its principal components. Comprehensive experiments have been conducted on several benchmark data sets, and the results demonstrate the effectiveness of the proposed algorithm.

A Novel Statistical Method for Scene Classification Based on Multi-Object Categorization and Logistic Regression

Abstract: In recent years, interest in scene classification of different indoor-outdoor scene images has increased due to major developments in visual sensor techniques. Scene classification has been demonstrated to be an efficient method for environmental observations but it is a challenging task considering the complexity of multiple objects in scenery images. These images include a combination of different properties and objects i.e., (color, text, and regions) and they are classified on the basis of optimal features. In this paper, an efficient multiclass objects categorization method is proposed for the indoor-outdoor scene classification of scenery images using benchmark datasets. We illustrate two improved methods, fuzzy c-mean and mean shift algorithms, which infer multiple object segmentation in complex images. Multiple object categorization is achieved through multiple kernel learning (MKL), which considers local descriptors and signatures of regions. The relations between multiple objects are then examined by intersection over union algorithm. Finally, scene classification is achieved by using Multi-class Logistic Regression (McLR). Experimental evaluation demonstrated that our scene classification method is superior compared to other conventional methods, especially when dealing with complex images. Our system should be applicable in various domains such as drone targeting, autonomous driving, Global positioning systems, robotics and tourist guide applications.

Learning Deep Kernels for Non-Parametric Two-Sample Tests

Abstract: We propose a class of kernel-based two-sample tests, which aim to determine whether two sets of samples are drawn from the same distribution. Our tests are constructed from kernels parameterized by deep neural nets, trained to maximize test power. These tests adapt to variations in distribution smoothness and shape over space, and are especially suited to high dimensions and complex data. By contrast, the simpler kernels used in prior kernel testing work are spatially homogeneous, and adaptive only in lengthscale. We explain how this scheme includes popular classifier-based two-sample tests as a special case, but improves on them in general. We provide the first proof of consistency for the proposed adaptation method, which applies both to kernels on deep features and to simpler radial basis kernels or multiple kernel learning. In experiments, we establish the superior performance of our deep kernels in hypothesis testing on benchmark and real-world data. The code of our deep-kernel-based two sample tests is available at this https URL.

Predicting Students’ Performance With School and Family Tutoring Using Generative Adversarial Network-Based Deep Support Vector Machine

Abstract: It has been witnessed that supportive learning has played a crucial role in educational quality enhancement. School and family tutoring offer personalized help and provide positive feedback on students’ learning. Predicting students’ performance is of much interest which reflects their understanding on the subjects. Particularly it is desired students to manage well in fundamental knowledge in order to build a strong foundation for post-secondary studies and career. In this paper, improved conditional generative adversarial network based deep support vector machine (ICGAN-DSVM) algorithm has been proposed to predict students’ performance under supportive learning via school and family tutoring. Owning to the nature of the students’ academic dataset is generally low sample size. ICGAN-DSVM offers dual benefits for the nature of low sample size in students’ academic dataset in which ICGAN increases the data volume whereas DSVM enhances the prediction accuracy with deep learning architecture. Results with 10-fold cross-validation show that the proposed ICGAN-DSVM yields specificity, sensitivity and area under the receiver operating characteristic curve (AUC) of 0.968, 0.971 and 0.954 respectively. Results also suggest that incorporating both school and family tutoring into the prediction model could further improve the performance compared with only school tutoring and only family tutoring. To show the necessity of ICGAN and DSVM, comparison has been made between ICGAN and traditional conditional generative adversarial network (CGAN). Also, the proposed kernel design via heuristic based multiple kernel learning (MKL) is compared with typical kernels including linear, radial basis function (RBF), polynomial and sigmoid. The prediction of student’s performance with and without GAN is presented which is followed by comparison with DSVM and with traditional SVM. The proposed ICGAN-DSVM outperforms related works by 8-29% in terms of performance indicators specificity, sensitivity and AUC.

Identification of drug–target interactions via fuzzy bipartite local model

Abstract: With the emergence of large-scale experimental data on genes and proteins, drug discovery and repositioning will be more difficult in the field of biomedical research. More and more resources are needed for detecting drug–target interactions (DTIs) in the experimental works. The interactions between drugs and targets could been seen as a bipartite network. Many computational methods have been developed to identify DTIs. However, most of them did not integrate multiple information and filter noise or outlier points. In this paper, we develop a fuzzy bipartite local model (FBLM) based on fuzzy least squares support vector machine and multiple kernel learning (MKL) for predicting DTIs. First, multiple kernels are constructed in drug and target spaces, respectively. Then, all corresponding kernels are combined by MKL algorithm in two spaces. Finally, FBLM is employed to identify DTIs. Our proposed approach is tested on four benchmark datasets under three types of cross validation. Comparing with existing outstanding methods, our method is a useful tool for the DTIs prediction.

Absent Multiple Kernel Learning Algorithms

Abstract: Multiple kernel learning (MKL) has been intensively studied during the past decade. It optimally combines the multiple channels of each sample to improve classification performance. However, existing MKL algorithms cannot effectively handle the situation where some channels of the samples are missing, which is not uncommon in practical applications. This paper proposes three absent MKL (AMKL) algorithms to address this issue. Different from existing approaches where missing channels are first imputed and then a standard MKL algorithm is deployed on the imputed data, our algorithms directly classify each sample based on its observed channels, without performing imputation. Specifically, we define a margin for each sample in its own relevant space, a space corresponding to the observed channels of that sample. The proposed AMKL algorithms then maximize the minimum of all sample-based margins, and this leads to a difficult optimization problem. We first provide two two-step iterative algorithms to approximately solve this problem. After that, we show that this problem can be reformulated as a convex one by applying the representer theorem. This makes it readily be solved via existing convex optimization packages. In addition, we provide a generalization error bound to justify the proposed AMKL algorithms from a theoretical perspective. Extensive experiments are conducted on nine UCI and six MKL benchmark datasets to compare the proposed algorithms with existing imputation-based methods. As demonstrated, our algorithms achieve superior performance and the improvement is more significant with the increase of missing ratio.

Generalized Tensor Regression for Hyperspectral Image Classification

Abstract: In this article, we propose a novel tensorial approach, namely, generalized tensor regression, for hyperspectral image classification. First, a simple and effective classifier, i.e., the ridge regression for multivariate labels, is extended to its tensorial version by taking advantages of tensorial representation. Then, the discrimination information of different modes is exploited to further strengthen the capacity of the model. Moreover, the model can be simplified and solved easily. Different from traditional tensorial methods, the proposed model can be utilized to capture not only the intrinsic structure of data in a physical sense but also the generalized relationship of data in a logical sense. Our proposed approach is shown to be effective for different classification purposes on a series of instantiations. Specifically, our experiment results with hyperspectral images collected by the airborne visible/infrared imaging spectrometer, the reflective optics spectrographic imaging system and the ITRES CASI-1500 demonstrate the effectiveness of the proposed approach as compared to other tensor-based classifiers and multiple kernel learning methods.

Multiple kernel subspace clustering with local structural graph and low-rank consensus kernel learning

Abstract: Multiple kernel learning (MKL) methods are generally believed to perform better than single kernel learning (SKL) methods in handling nonlinear subspace clustering problem, largely thanks to MKL avoids selecting and tuning a pre-defined kernel. However, previous MKL methods mainly focused on how to define a kernel weighting strategy, but ignored the structural characteristics of the input data in both the original space and the kernel space. In this paper, we first propose a novel graph-based MKL method for subspace clustering, namely, Local Structural Graph and Low-Rank Consensus Multiple Kernel Learning (LLMKL). It jointly learns an optimal affinity graph and a suitable consensus kernel for clustering purpose by elegantly integrating the MKL technology, the global structure in the kernel space, the local structure in the original space, and the Hilbert space self-expressiveness property in a unified optimization model. In particular, to capture the data global structure, we employ a substitute of the desired consensus kernel, and then introduce a low-rank constraint on the substitute to encourage that the structure of linear subspaces is present in the feature space. Moreover, the data local structure is explored by building a complete graph, where each sample is treated as a node, and an edge codes the pairwise affinity between two samples. By such, the consensus kernel learning and the affinity graph learning can promote each other such that the data in resulting Hilbert space are both self-expressive and low-rank. Experiments on both image and text clustering well demonstrate that LLMKL outperforms the state-of-the-art methods.

Histogram of Oriented Gradient-Based Fusion of Features for Human Action Recognition in Action Video Sequences.

Abstract: Human Action Recognition (HAR) is the classification of an action performed by a human. The goal of this study was to recognize human actions in action video sequences. We present a novel feature descriptor for HAR that involves multiple features and combining them using fusion technique. The major focus of the feature descriptor is to exploits the action dissimilarities. The key contribution of the proposed approach is to built robust features descriptor that can work for underlying video sequences and various classification models. To achieve the objective of the proposed work, HAR has been performed in the following manner. First, moving object detection and segmentation are performed from the background. The features are calculated using the histogram of oriented gradient (HOG) from a segmented moving object. To reduce the feature descriptor size, we take an averaging of the HOG features across non-overlapping video frames. For the frequency domain information we have calculated regional features from the Fourier hog. Moreover, we have also included the velocity and displacement of moving object. Finally, we use fusion technique to combine these features in the proposed work. After a feature descriptor is prepared, it is provided to the classifier. Here, we have used well-known classifiers such as artificial neural networks (ANNs), support vector machine (SVM), multiple kernel learning (MKL), Meta-cognitive Neural Network (McNN), and the late fusion methods. The main objective of the proposed approach is to prepare a robust feature descriptor and to show the diversity of our feature descriptor. Though we are using five different classifiers, our feature descriptor performs relatively well across the various classifiers. The proposed approach is performed and compared with the state-of-the-art methods for action recognition on two publicly available benchmark datasets (KTH and Weizmann) and for cross-validation on the UCF11 dataset, HMDB51 dataset, and UCF101 dataset. Results of the control experiments, such as a change in the SVM classifier and the effects of the second hidden layer in ANN, are also reported. The results demonstrate that the proposed method performs reasonably compared with the majority of existing state-of-the-art methods, including the convolutional neural network-based feature extractors.

Learning Deep Kernels for Non-Parametric Two-Sample Tests

Abstract: We propose a class of kernel-based two-sample tests, which aim to determine whether two sets of samples are drawn from the same distribution. Our tests are constructed from kernels parameterized by deep neural nets, trained to maximize test power. These tests adapt to variations in distribution smoothness and shape over space, and are especially suited to high dimensions and complex data. By contrast, the simpler kernels used in prior kernel testing work are spatially homogeneous, and adaptive only in lengthscale. We explain how this scheme includes popular classifier-based two-sample tests as a special case, but improves on them in general. We provide the first proof of consistency for the proposed adaptation method, which applies both to kernels on deep features and to simpler radial basis kernels or multiple kernel learning. In experiments, we establish the superior performance of our deep kernels in hypothesis testing on benchmark and real-world data.

A Generic Design of Driver Drowsiness and Stress Recognition Using MOGA Optimized Deep MKL-SVM

Abstract: Driver drowsiness and stress are major causes of traffic deaths and injuries, which ultimately wreak havoc on world economic loss. Researchers are in full swing to develop various algorithms for both drowsiness and stress recognition. In contrast to existing works, this paper proposes a generic model using multiple-objective genetic algorithm optimized deep multiple kernel learning support vector machine that is capable to recognize both driver drowsiness and stress. This algorithm simplifies the research formulations and model complexity that one model fits two applications. Results reveal that the proposed algorithm achieves an average sensitivity of 99%, specificity of 98.3% and area under the receiver operating characteristic curve (AUC) of 97.1% for driver drowsiness recognition. For driver stress recognition, the best performance is yielded with average sensitivity of 98.7%, specificity of 98.4% and AUC of 96.9%. Analysis also indicates that the proposed algorithm using multiple-objective genetic algorithm has better performance compared to the grid search method. Multiple kernel learning enhances the performance significantly compared to single typical kernel. Compared with existing works, the proposed algorithm not only achieves higher accuracy but also addressing the typical issues of dataset in simulated environment, no cross-validation and unreliable measurement stability of input signals.

Deep Learning Based Analysis of Breast Cancer Using Advanced Ensemble Classifier and Linear Discriminant Analysis

Abstract: In the recent past, the Classifiers are based on genetic signatures in which many microarray studies are analyzed to predict medical results for cancer patients However, the Signatures from different studies have been benefitted with low-intensity ratio during the classification of individual datasets has been considered as a significant point of research in the present scenario Hence to overcome the above-discussed issue, this paper provides a Deep Learning Framework that combines an algorithm of necessary processing of Linear Discriminant Analysis (LDA) and Auto Encoder (AE) Neural Network to classify different features within the profile of gene expression Hence, an advanced ensemble classification has been developed based on the Deep Learning (DL) algorithm to assess the clinical outcome of breast cancer Furthermore, numerous independent breast cancer datasets and representations of the signature gene, including the primary method, have been evaluated for the optimization parameters Finally, the experiment results show that the suggested deep learning frameworks achieve 9827% accuracy than many other techniques such as genomic data and pathological images with multiple kernel learning (GPMKL), Multi-Layer Perception (MLP), Deep Learning Diagnosis (DLD), and Spatiotemporal Wavelet Kinetics (SWK)

Multisource Domain Attribute Adaptation Based on Adaptive Multikernel Alignment Learning

Abstract: For attribute-based zero-shot learning (ZSL), the attribute classifiers learned previously on the training images may not be usable for the testing images due to that the training and testing images may follow different data distributions. Since domain adaptation learning can effectively perform knowledge transfer under the circumstance of different data distributions, we proposed a novel ZSL method, referred to as multisource domain attribute adaptation based on adaptive multikernel alignment learning (A-MKAL), from the point of view of classifier adaptation. Considering there may be a large difference between object classes, we adopt the clustering method to group the training images according to the class–class correlation measured by the whitened cosine similarity, thus multiple source domains are created. The created multiple source domains are then combined into one weighted source domain to participate in the distribution discrepancy match across domains. In order to adapt the attribute classifier learned on the well-defined source domains to the target domain (the training image set), we designed the A-MKAL by applying the centered kernel alignment to align the attribute kernel matrix and the kernel function of adaptive multiple kernel learning. Experiments on Shoes, OSR, and AWA datasets show that, compared with state-of-the-art methods, our proposed method yields more accurate classification.

Multiple kernel learning for integrative consensus clustering of omic datasets.

Abstract: MOTIVATION Diverse applications-particularly in tumour subtyping-have demonstrated the importance of integrative clustering techniques for combining information from multiple data sources. Cluster Of Clusters Analysis (COCA) is one such approach that has been widely applied in the context of tumour subtyping. However, the properties of COCA have never been systematically explored, and its robustness to the inclusion of noisy datasets is unclear. RESULTS We rigorously benchmark COCA, and present Kernel Learning Integrative Clustering (KLIC) as an alternative strategy. KLIC frames the challenge of combining clustering structures as a multiple kernel learning problem, in which different datasets each provide a weighted contribution to the final clustering. This allows the contribution of noisy datasets to be down-weighted relative to more informative datasets. We compare the performances of KLIC and COCA in a variety of situations through simulation studies. We also present the output of KLIC and COCA in real data applications to cancer subtyping and transcriptional module discovery. AVAILABILITY AND IMPLEMENTATION R packages klic and coca are available on the Comprehensive R Archive Network. SUPPLEMENTARY INFORMATION Supplementary data are available at Bioinformatics online.

A Novel Triple Matrix Factorization Method for Detecting Drug-Side Effect Association Based on Kernel Target Alignment.

Abstract: All drugs usually have side effects, which endanger the health of patients. To identify potential side effects of drugs, biological and pharmacological experiments are done but are expensive and time-consuming. So, computation-based methods have been developed to accurately and quickly predict side effects. To predict potential associations between drugs and side effects, we propose a novel method called the Triple Matrix Factorization- (TMF-) based model. TMF is built by the biprojection matrix and latent feature of kernels, which is based on Low Rank Approximation (LRA). LRA could construct a lower rank matrix to approximate the original matrix, which not only retains the characteristics of the original matrix but also reduces the storage space and computational complexity of the data. To fuse multivariate information, multiple kernel matrices are constructed and integrated via Kernel Target Alignment-based Multiple Kernel Learning (KTA-MKL) in drug and side effect space, respectively. Compared with other methods, our model achieves better performance on three benchmark datasets. The values of the Area Under the Precision-Recall curve (AUPR) are 0.677, 0.685, and 0.680 on three datasets, respectively.

Forecasting air pollution PM2.5 in Beijing using weather data and multiple kernel learning

Abstract: PM2.5 mass concentration prediction is an important research issue because of the increasing impact of air pollution on the urban environment. In this paper, a PM2.5 forecasting framework incorporating meteorological factors based on multiple kernel learning (MKL) is proposed to forecast the near future PM2.5. In addition, we develop a novel two‐step algorithm for solving the primal MKL problem. Compared with most existing MKL 2‐step algorithms, the proposed algorithm does not require the optimal step size for updating kernel combination coefficients by linear search. To demonstrate the performance of the proposed forecasting framework, its performance is compared to single kernel‐based support vector regression (SVR). Data sets of an inland city Beijing acquired from UCI are used to train and validate both of two methods. Experiments show that our proposed method outperforms the SVR.

MVIIDroid: A Multiple View Information Integration Approach for Android Malware Detection and Family Identification

Abstract: With the rapid growth of Android applications, there is an urgent need for powerful Android malware detection technology nowadays. Existing classification models can be summarized with the following two steps-feature extraction and classification model learning. To further enhance the representation ability of existing classification models, this article presents an Android malicious application detection framework termed multiview information integration technology (MVIIDroid). To be specific, in our approach, we extract applications’ multiple components, transform them into embedding feature vectors and train a multiple Kernel learning model as the classifier. To illustrate the effectiveness of our model, we evaluate MVIIDroid on two Android malware datasets of 6820 malware and 6820 benign applications. Results show that we have superior classification performances when separating malware from benign applications. Moreover, we further evaluate MVIIDroid's ability to attribute malicious applications to their actual families. The experimental results well demonstrate the effectiveness of the proposed model.

Learning adaptive contrast combinations for visual saliency detection

Abstract: Visual saliency detection plays a significant role in the fields of computer vision. In this paper, we introduce a novel saliency detection method based on weighted linear multiple kernel learning (WLMKL) framework, which is able to adaptively combine different contrast measurements in a supervised manner. As most influential factor is contrast operation in bottom-up visual saliency, an average weighted corner-surround contrast (AWCSC) is first designed to measure local visual saliency. Combined with common-used center-surrounding contrast (CESC) and global contrast (GC), three types of contrast operations are fed into our WLMKL framework to produce the final saliency map. We show that the assigned weights for each contrast feature maps are always normalized in our WLMKL formulation. In addition, the proposed approach benefits from the advantages of the contribution of each individual contrast feature maps, yielding more robust and accurate saliency maps. We evaluated our method for two main visual saliency detection tasks: human fixed eye prediction and salient object detection. The extensive experimental results show the effectiveness of the proposed model, and demonstrate the integration is superior than individual subcomponent.

Convex Optimization for Group Feature Selection in Networked Data

Abstract: Feature selection is at the heart of machine learning, and it is effective at facilitating data interpretability and improving prediction performance by defying the curse of dimensionality. Group f...

MCK-ELM: multiple composite kernel extreme learning machine for hyperspectral images

Abstract: Multiple kernel (MK) learning (MKL) methods have a significant impact on improving the classification performance. Besides that, composite kernel (CK) methods have high capability on the analysis of hyperspectral images due to making use of the contextual information. In this work, it is aimed to aggregate both CKs and MKs autonomously without the need of kernel coefficient adjustment manually. Convex combination of predefined kernel functions is implemented by using multiple kernel extreme learning machine. Thus, complex optimization processes of standard MKL are disposed of and the facility of multi-class classification is profited. Different types of kernel functions are placed into MKs in order to realize hybrid kernel scenario. The proposed methodology is performed over Pavia University, Indian Pines, and Salinas hyperspectral scenes that have ground-truth information. Multiple composite kernels are constructed using Gaussian, polynomial, and logarithmic kernel functions with various parameters, and then the obtained results are presented comparatively along with the state-of-the-art standard machine learning, MKL, and CK methods.

Ship Target Discrimination in SAR Images Based on BOW Model With Multiple Features and Spatial Pyramid Matching

Abstract: To eliminate the false alarms in the ship target detection effectively for synthetic aperture radar (SAR) images in complex scenes, this article present a novel ship target discrimination algorithm based on bag of words (BOW) model with multiple features and spatial pyramid matching (SPM), which is named MF-SPM-BOW. The proposed discrimination method mainly contains three stages. First, the SAR scale-invariant feature transform (SAR-SIFT) descriptors and gray-level co-occurrence matrix (GLCM) descriptors are extracted as local features to describe the gradient information and texture information of local regions of an image chip. Then, the SPM technique considering its spatial location information-keeping capability is employed to generate global features with excellent discrimination ability. Finally, the support vector machine (SVM) discriminator based on multiple kernel learning is applied to realize feature fusion in image layer and thus identify targets and clutter. Experimental results show that compared with the traditional discrimination methods and the BOW model discrimination methods, the proposed SAR ship target discrimination algorithm achieves better discrimination performance, which can eliminate most of the false alarms in candidate ship target chips effectively.