Showing papers on "Multiple kernel learning published in 2019"

Transferable Representation Learning with Deep Adaptation Networks

Abstract: Domain adaptation studies learning algorithms that generalize across source domains and target domains that exhibit different distributions Recent studies reveal that deep neural networks can learn transferable features that generalize well to similar novel tasks However, as deep features eventually transition from general to specific along the network, feature transferability drops significantly in higher task-specific layers with increasing domain discrepancy To formally reduce the effects of this discrepancy and enhance feature transferability in task-specific layers, we develop a novel framework for deep adaptation networks that extends deep convolutional neural networks to domain adaptation problems The framework embeds the deep features of all task-specific layers into reproducing kernel Hilbert spaces (RKHSs) and optimally matches different domain distributions The deep features are made more transferable by exploiting low-density separation of target-unlabeled data in very deep architectures, while the domain discrepancy is further reduced via the use of multiple kernel learning that enhances the statistical power of kernel embedding matching The overall framework is cast in a minimax game setting Extensive empirical evidence shows that the proposed networks yield state-of-the-art results on standard visual domain-adaptation benchmarks

Low-rank Kernel Learning for Graph-based Clustering

Abstract: Constructing the adjacency graph is fundamental to graph-based clustering. Graph learning in kernel space has shown impressive performance on a number of benchmark data sets. However, its performance is largely determined by the chosen kernel matrix. To address this issue, previous multiple kernel learning algorithm has been applied to learn an optimal kernel from a group of predefined kernels. This approach might be sensitive to noise and limits the representation ability of the consensus kernel. In contrast to existing methods, we propose to learn a low-rank kernel matrix which exploits the similarity nature of the kernel matrix and seeks an optimal kernel from the neighborhood of candidate kernels. By formulating graph construction and kernel learning in a unified framework, the graph and consensus kernel can be iteratively enhanced by each other. Extensive experimental results validate the efficacy of the proposed method.

Identification of Drug-Side Effect Association via Semisupervised Model and Multiple Kernel Learning

Abstract: Drug-side effect association contains the information on marketed medicines and their recorded adverse drug reactions. Traditional experimental method is time consuming and expensive. All associations of drugs and side-effects are seen as a bipartite network. Therefore, many computational approaches have been developed to deal with this problem, which are used to predict new potential associations. However, lots of methods did not consider multiple kernel learning (MKL) algorithm, which can integrate multiple sources of information and further improve prediction performance. In this study, we develop a novel predictor of drug-side effect association. First, we build multiple kernels from drug space and side-effect space. What is more, these corresponding kernels are linear weighted by MKL algorithm in drug space and side-effect space, respectively. Finally, a graph-based semisupervised learning is employed to construct drug-side effect predictor. Compared with existing methods, our method achieves better results on three benchmark data sets. The values of area under the precision recall curve are 0.668, 0.673, and 0.670 on three benchmark data sets, respectively. Our method is a useful tool for the side-effects prediction of drugs.

Heterogeneous defect prediction with two-stage ensemble learning

Abstract: Heterogeneous defect prediction (HDP) refers to predicting defect-prone software modules in one project (target) using heterogeneous data collected from other projects (source). Recently, several HDP methods have been proposed. However, these methods do not sufficiently incorporate the two characteristics of the defect data: (1) data could be linear inseparable, and (2) data could be highly imbalanced. These two data characteristics make it challenging to build an effective HDP model. In this paper, we propose a novel Two-Stage Ensemble Learning (TSEL) approach to HDP, which contains two stages: ensemble multi-kernel domain adaptation (EMDA) stage and ensemble data sampling (EDS) stage. In the EMDA stage, we develop an Ensemble Multiple Kernel Correlation Alignment (EMKCA) predictor, which combines the advantage of multiple kernel learning and domain adaptation techniques. In the EDS stage, we employ RESample with replacement (RES) technique to learn multiple different EMKCA predictors and use average ensemble to combine them together. These two stages create an ensemble of defect predictors. Extensive experiments on 30 public projects show that the proposed TSEL approach outperforms a range of competing methods. The improvement is 20.14–33.92% in AUC, 36.05–54.78% in f-measure, and 5.48–19.93% in balance, respectively.

Learning Multilevel Auto-Encoders for DDoS Attack Detection in Smart Grid Network

Abstract: Bidirectional communication infrastructure of smart systems, such as smart grids, are vulnerable to network attacks like distributed denial of services (DDoS) and can be a major concern in the present competitive market. In DDoS attack, multiple compromised nodes in a communication network flood connection requests, bogus data packets or incoming messages to targets like database servers, resulting in denial of services for legitimate users. Recently, machine learning based techniques have been explored by researchers to secure the network from DDoS attacks. Under different attack scenarios on a system, measurements can be observed either in an online manner or batch mode and can be used to build predictive learning systems. In this work, we propose an efficient DDoS attack detection technique based on multilevel auto-encoder based feature learning. We learn multiple levels of shallow and deep auto-encoders in an unsupervised manner which are then used to encode the training and test data for feature generation. A final unified detection model is then learned by combining the multilevel features using and efficient multiple kernel learning (MKL) algorithm. We perform experiments on two benchmark DDoS attack databases and their subsets and compare the results with six recent methods. Results show that the proposed method outperforms the compared methods in terms of prediction accuracy.

Classifying Breast Cancer Subtypes Using Multiple Kernel Learning Based on Omics Data

Abstract: It is very significant to explore the intrinsic differences in breast cancer subtypes. These intrinsic differences are closely related to clinical diagnosis and designation of treatment plans. With the accumulation of biological and medicine datasets, there are many different omics data that can be viewed in different aspects. Combining these multiple omics data can improve the accuracy of prediction. Meanwhile; there are also many different databases available for us to download different types of omics data. In this article, we use estrogen receptor (ER), progesterone receptor (PR), human epidermal growth factor receptor 2 (HER2) to define breast cancer subtypes and classify any two breast cancer subtypes using SMO-MKL algorithm. We collected mRNA data, methylation data and copy number variation (CNV) data from TCGA to classify breast cancer subtypes. Multiple Kernel Learning (MKL) is employed to use these omics data distinctly. The result of using three omics data with multiple kernels is better than that of using single omics data with multiple kernels. Furthermore; these significant genes and pathways discovered in the feature selection process are also analyzed. In experiments; the proposed method outperforms other state-of-the-art methods and has abundant biological interpretations.

Multi-Kernel Coupled Projections for Domain Adaptive Dictionary Learning

Abstract: Dictionary learning has produced state-of-the-art results in various classification tasks. However, if the training data have a different distribution than the testing data, the learned sparse representation might not be optimal. Recently, several domain-adaptive dictionary learning (DADL) methods and kernels have been proposed and have achieved impressive performance. However, the performance of these single kernel-based methods heavily depends heavily on the choice of the kernel, and the question of how to combine multiple kernel learning (MKL) with the DADL framework has not been well studied. Motivated by these concerns, in this paper, we propose a multi-kernel domain-adaptive sparse representation-based classification (MK-DASRC) and then use it as a criterion to design a multi-kernel sparse representation-based domain-adaptive discriminative projection method, in which the discriminative features of the data in the two domains are simultaneously learned with the dictionary. The purpose of this method is to maximize the between-class sparse reconstruction residuals of data from both domains, and minimize the within-class sparse reconstruction residuals of data in the low-dimensional subspace. Thus, the resulting representations can satisfactorily fit MK-DASRC and simultaneously display discriminability. Extensive experimental results on a series of benchmark databases show that our method performs better than the state-of-the-art methods.

Localized Multiple Kernel Learning for Anomaly Detection: One-class Classification

Abstract: Multi-kernel learning has been well explored in the recent past and has exhibited promising outcomes for multi-class classification and regression tasks. In this paper, we present a multiple kernel learning approach for the One-class Classification (OCC) task and employ it for anomaly detection. Recently, the basic multi-kernel approach has been proposed to solve the OCC problem, which is simply a convex combination of different kernels with equal weights. This paper proposes a Localized Multiple Kernel learning approach for Anomaly Detection ( L M K A D ) using OCC, where the weight for each kernel is assigned locally. Proposed L M K A D approach adapts the weight for each kernel using a gating function. The parameters of the gating function and one-class classifier are optimized simultaneously through a two-step optimization process. We present the empirical results of the performance of L M K A D on 25 benchmark datasets from various disciplines. This performance is evaluated against existing Multi Kernel Anomaly Detection ( M K A D ) algorithm, and four other existing kernel-based one-class classifiers to showcase the credibility of our approach. L M K A D achieves significantly better Gmean scores while using a lesser number of support vectors compared to M K A D . Friedman test is also performed to verify the statistical significance of the results claimed in this paper.

A hybrid approach to building face shape classifier for hairstyle recommender system

Abstract: Identifying human face shape is the first and the most vital process prior to choosing the right hairstyle to wear on according to guidelines from hairstyle experts, especially for women. This work presents a novel framework for a hairstyle recommender system that is based on face shape classifier. This framework enables an automatic hairstyle recommendation with a single face image. This has a direct impact on beauty industry service providers. It can simulate how the user looks like when she is wearing the chosen hairstyle recommended by the expert system. The model used in this framework is based on Support Vector Machine. The framework is evaluated on hand-crafted, deep-learned (VGG-face) features and VGG-face fine-tuned version for the face shape classification task. In addition to evaluating these individual features by a well-designed framework, we attempted to fuse these three descriptors together in order to improve the performance of the classification task. Two combination techniques were employed, namely: Vector Concatenation and Multiple Kernel Learning (MKL) techniques. All the hyper-parameters of the model were optimised by using Particle Swarm Optimisation. The results show that combining hand-crafted and VGG-face descriptors with MKL yielded the best results at 70.3% of accuracy which was statistically significantly better than using individual features. Thus, combining multiple representations of the data with MKL can improve the overall performance of the expert system. In addition, this proves that hand-crafted descriptor can be complementary to deep-learned descriptor.

Effects of Compression on Remote Sensing Image Classification Based on Fractal Analysis

Abstract: Image compression is essential for remote sensing due to the large volume of produced remote sensing imagery and system’s limited transmission or storage capacity. As one of the most important applications, classification might be affected due to the introduced distortion during compression. Hence, we perform a quantitative study on the effects of compression on remote sensing image classification and propose a method to estimate the remote sensing image classification accuracy based on fractal analysis. Multiscale feature extraction is performed and a multiple kernel learning approach is proposed accordingly. The experimental results on our established database indicate that the classification accuracy predicted by our method exhibits high consistency with the ground truth and our method shows its superiority when compared with other classical reference algorithms.

Multiple-kernel learning for genomic data mining and prediction

Abstract: Advances in medical technology have allowed for customized prognosis, diagnosis, and treatment regimens that utilize multiple heterogeneous data sources. Multiple kernel learning (MKL) is well suited for the integration of multiple high throughput data sources. MKL remains to be under-utilized by genomic researchers partly due to the lack of unified guidelines for its use, and benchmark genomic datasets. We provide three implementations of MKL in R. These methods are applied to simulated data to illustrate that MKL can select appropriate models. We also apply MKL to combine clinical information with miRNA gene expression data of ovarian cancer study into a single analysis. Lastly, we show that MKL can identify gene sets that are known to play a role in the prognostic prediction of 15 cancer types using gene expression data from The Cancer Genome Atlas, as well as, identify new gene sets for the future research. Multiple kernel learning coupled with modern optimization techniques provides a promising learning tool for building predictive models based on multi-source genomic data. MKL also provides an automated scheme for kernel prioritization and parameter tuning. The methods used in the paper are implemented as an R package called RMKL package, which is freely available for download through CRAN at https://CRAN.R-project.org/package=RMKL .

Parcel-Based Crop Classification Using Multi-Temporal TerraSAR-X Dual Polarimetric Data

Abstract: Cropland maps are useful for the management of agricultural fields and the estimation of harvest yield. Some local governments have documented field properties, including crop type and location, based on site investigations. This process, which is generally done manually, is labor-intensive, and remote-sensing techniques can be used as alternatives. In this study, eight crop types (beans, beetroot, grass, maize, potatoes, squash, winter wheat, and yams) were identified using gamma naught values and polarimetric parameters calculated from TerraSAR-X (or TanDEM-X) dual-polarimetric (HH/VV) data. Three indices (difference (D-type), simple ratio (SR), and normalized difference (ND)) were calculated using gamma naught values and m-chi decomposition parameters and were evaluated in terms of crop classification. We also evaluated the classification accuracy of four widely used machine-learning algorithms (kernel-based extreme learning machine, support vector machine, multilayer feedforward neural network (FNN), and random forest) and two multiple-kernel methods (multiple kernel extreme learning machine (MKELM) and multiple kernel learning (MKL)). MKL performed best, achieving an overall accuracy of 92.1%, and proved useful for the identification of crops with small sample sizes. The difference (raw or normalized) between double-bounce scattering and odd-bounce scattering helped to improve the identification of squash and yams fields.

Online Asymmetric Metric Learning With Multi-Layer Similarity Aggregation for Cross-Modal Retrieval

Abstract: Cross-modal retrieval has attracted intensive attention in recent years, where a substantial yet challenging problem is how to measure the similarity between heterogeneous data modalities. Despite using modality-specific representation learning techniques, most existing shallow or deep models treat different modalities equally and neglect the intrinsic modality heterogeneity and information imbalance among images and texts. In this paper, we propose an online similarity function learning framework to learn the metric that can well reflect the cross-modal semantic relation. Considering that multiple CNN feature layers naturally represent visual information from low-level visual patterns to high-level semantic abstraction, we propose a new asymmetric image–text similarity formulation which aggregates the layer-wise visual–textual similarities parameterized by different bilinear parameter matrices. To effectively learn the aggregated similarity function, we develop three different similarity combination strategies, i.e., average kernel, multiple kernel learning, and layer gating. The former two kernel-based strategies assign uniform weights on different layers to all data pairs; the latter works on the original feature representation and assigns instance-aware weights on different layers to different data pairs, and they are all learned by preserving the bi-directional relative similarity expressed by a large number of cross-modal training triplets. The experiments conducted on three public datasets well demonstrate the effectiveness of our methods.

Multivariate Information Fusion With Fast Kernel Learning to Kernel Ridge Regression in Predicting LncRNA-Protein Interactions.

Abstract: Long non-coding RNAs (lncRNAs) constitute a large class of transcribed RNA molecules. They have a characteristic length of more than 200 nucleotides which do not encode proteins. They play an important role in regulating gene expression by interacting with the homologous RNA-binding proteins. Due to the laborious and time-consuming nature of wet experimental methods, more researchers should pay great attention to computational approaches for the prediction of lncRNA-protein interaction (LPI). An in-depth literature review in the state-of-the-art in silico investigations, leads to the conclusion that there is still room for improving the accuracy and velocity. This paper propose a novel method for identifying LPI by employing Kernel Ridge Regression, based on Fast Kernel Learning (LPI-FKLKRR). This approach, uses four distinct similarity measures for lncRNA and protein space, respectively. It is remarkable, that we extract Gene Ontology (GO) with proteins, in order to improve the quality of information in protein space. The process of heterogeneous kernels integration, applies Fast Kernel Learning (FastKL) to deal with weight optimization. The extrapolation model is obtained by gaining the ultimate prediction associations, after using Kernel Ridge Regression (KRR). Experimental outcomes show that the ability of modeling with LPI-FKLKRR has extraordinary performance compared with LPI prediction schemes. On benchmark dataset, it has been observed that the best Area Under Precision Recall Curve (AUPR) of 0.6950 is obtained by our proposed model LPI-FKLKRR, which outperforms the integrated LPLNP (AUPR: 0.4584), RWR (AUPR: 0.2827), CF (AUPR: 0.2357), LPIHN (AUPR: 0.2299), and LPBNI (AUPR: 0.3302). Also, combined with the experimental results of a case study on a novel dataset, it is anticipated that LPI-FKLKRR will be a useful tool for LPI prediction.

A Bayesian Multiple Kernel Learning Algorithm for SSVEP BCI Detection

Abstract: This paper deals with the classification of steady-state visual evoked potentials (SSVEP), which is a multiclass classification problem addressed in SSVEP-based brain–computer interfaces. In particular, our method named MultiLRM_MKL uses multiple linear regression models under a Sparse Bayesian Learning (SBL) framework to discriminate between the SSVEP classes. The regression coefficients of each model are learned using the Variational Bayesian (VB) framework and the kernel trick is adopted not only for reducing the computational cost of our method, but also for enabling the combination of different kernel spaces. We verify the effectiveness of our method in handling different kernel spaces by evaluating its performance with a new kernel based on canonical correlation analysis. In particular, we prove the benefit of combining multiple kernels by outperforming several state-of-the-art methods in two SSVEP datasets, reaching an information transfer rate of 93 b/min using only three channels from the occipital area ( $O_z$ , $O_1$ , and $O_2$ ).

Visual complexity modelling based on image features fusion of multiple kernels.

Abstract: Humans' perception of visual complexity is often regarded as one of the key principles of aesthetic order, and is intimately related to the physiological, neurological and, possibly, psychological characteristics of the human mind For these reasons, creating accurate computational models of visual complexity is a demanding task Building upon on previous work in the field (Forsythe et al, 2011; Machado et al, 2015) we explore the use of Machine Learning techniques to create computational models of visual complexity For that purpose, we use a dataset composed of 800 visual stimuli divided into five categories, describing each stimulus by 329 features based on edge detection, compression error and Zipf's law In an initial stage, a comparative analysis of representative state-of-the-art Machine Learning approaches is performed Subsequently, we conduct an exhaustive outlier analysis We analyze the impact of removing the extreme outliers, concluding that Feature Selection Multiple Kernel Learning obtains the best results, yielding an average correlation to humans' perception of complexity of 071 with only twenty-two features These results outperform the current state-of-the-art, showing the potential of this technique for regression

FKRR-MVSF: A Fuzzy Kernel Ridge Regression Model for Identifying DNA-Binding Proteins by Multi-View Sequence Features via Chou's Five-Step Rule.

Abstract: DNA-binding proteins play an important role in cell metabolism. In biological laboratories, the detection methods of DNA-binding proteins includes yeast one-hybrid methods, bacterial singles and X-ray crystallography methods and others, but these methods involve a lot of labor, material and time. In recent years, many computation-based approachs have been proposed to detect DNA-binding proteins. In this paper, a machine learning-based method, which is called the Fuzzy Kernel Ridge Regression model based on Multi-View Sequence Features (FKRR-MVSF), is proposed to identifying DNA-binding proteins. First of all, multi-view sequence features are extracted from protein sequences. Next, a Multiple Kernel Learning (MKL) algorithm is employed to combine multiple features. Finally, a Fuzzy Kernel Ridge Regression (FKRR) model is built to detect DNA-binding proteins. Compared with other methods, our model achieves good results. Our method obtains an accuracy of 83.26% and 81.72% on two benchmark datasets (PDB1075 and compared with PDB186), respectively.

PIMKL: Pathway-Induced Multiple Kernel Learning.

Abstract: Reliable identification of molecular biomarkers is essential for accurate patient stratification. While state-of-the-art machine learning approaches for sample classification continue to push boundaries in terms of performance, most of these methods are not able to integrate different data types and lack generalization power, limiting their application in a clinical setting. Furthermore, many methods behave as black boxes, and we have very little understanding about the mechanisms that lead to the prediction. While opaqueness concerning machine behavior might not be a problem in deterministic domains, in health care, providing explanations about the molecular factors and phenotypes that are driving the classification is crucial to build trust in the performance of the predictive system. We propose Pathway-Induced Multiple Kernel Learning (PIMKL), a methodology to reliably classify samples that can also help gain insights into the molecular mechanisms that underlie the classification. PIMKL exploits prior knowledge in the form of a molecular interaction network and annotated gene sets, by optimizing a mixture of pathway-induced kernels using a Multiple Kernel Learning (MKL) algorithm, an approach that has demonstrated excellent performance in different machine learning applications. After optimizing the combination of kernels to predict a specific phenotype, the model provides a stable molecular signature that can be interpreted in the light of the ingested prior knowledge and that can be used in transfer learning tasks.

Superpixel Tensor Model for Spatial–Spectral Classification of Remote Sensing Images

Abstract: Nowadays, many methods of spatial–spectral classification have been developed and achieved good results for classification with high-resolution remotely sensed images, especially superpixel-based methods. However, these methods generally consider a superpixel as a group of pixels instead of one entity, ignoring the spectral–spatial entirety in the third-order RSI data cube. In order to fully exploit the third-order spectral–spatial information, in this paper, we propose a superpixel-based tensor model for RSI classification, where a multiattribute superpixel tensor (MAST) model is constructed on the top of multiattribute superpixel maps based on the concept of extended morphological profiles (EMAPs). In order to manage the adaptive spatial nature of superpixels, we develop an increment strategy to augment all superpixels with filling up their own envelop rectangles including three different ways, i.e., 0 vector, mean vector of all the pixels within the superpixel, or original pixels. Then, we use CANDECOMP/PARAFAC (CP) decomposition to obtain the features of the unified dimension from the MASTs of various sizes. Especially, CP decomposition can deal with missing data, so we also got a fourth means of constructing the MAST. Finally, base kernels calculated, respectively, from the original spectral feature, EMAP features and MAST features are learned by multiple kernel learning methods, with the optimal kernel fed to a support vector machine to complete the classification task. The experiments conducted on four real RSIs and compared with several well-known methods demonstrate the effectiveness of the proposed model.