Multiple kernel learning

About: Multiple kernel learning is a research topic. Over the lifetime, 1630 publications have been published within this topic receiving 56082 citations.

A novel multiple kernel learning approach for semi-supervised clustering

Abstract: Distance metrics are widely used in various machine learning and pattern recognition algorithms. A main issue in these algorithms is choosing the proper distance metric. In recent years, learning an appropriate distance metric has become a very active research field. In the kernelised version of distance metric learning algorithms, the data points are implicitly mapped into a higher dimensional feature space and the learning process is performed in the resulted feature space. The performance of the kernel-based methods heavily depends on the chosen kernel function. So, selecting an appropriate kernel function and/or tuning its parameter(s) impose significant challenges in such methods. The Multiple Kernel Learning theory (MKL) addresses this problem by learning a linear combination of a number of predefined kernels. In this paper, we formulate the MKL problem in a semi-supervised metric learning framework. In the proposed approach, pairwise similarity constraints are used to adjust the weights of the combined kernels and simultaneously learn the appropriate distance metric. Using both synthetic and real-world datasets, we show that the proposed method outperforms some recently introduced semi-supervised metric learning approaches.

Multiple Kernel $k$-Means Clustering by Selecting Representative Kernels

Abstract: To cluster data that are not linearly separable in the original feature space, $k$-means clustering was extended to the kernel version. However, the performance of kernel $k$-means clustering largely depends on the choice of kernel function. To mitigate this problem, multiple kernel learning has been introduced into the $k$-means clustering to obtain an optimal kernel combination for clustering. Despite the success of multiple kernel $k$-means clustering in various scenarios, few of the existing work update the combination coefficients based on the diversity of kernels, which leads to the result that the selected kernels contain high redundancy and would degrade the clustering performance and efficiency. In this paper, we propose a simple but efficient strategy that selects a diverse subset from the pre-specified kernels as the representative kernels, and then incorporate the subset selection process into the framework of multiple $k$-means clustering. The representative kernels can be indicated as the significant combination weights. Due to the non-convexity of the obtained objective function, we develop an alternating minimization method to optimize the combination coefficients of the selected kernels and the cluster membership alternatively. We evaluate the proposed approach on several benchmark and real-world datasets. The experimental results demonstrate the competitiveness of our approach in comparison with the state-of-the-art methods.

Learning in Reproducing Kernel Hilbert Spaces

Abstract: This chapter is dedicated to nonparametric modeling of nonlinear functions in reproducing kernel Hilbert spaces (RKHS). The basic definitions and concepts behind RKH spaces are presented, including positive definite kernels, reproducing kernels, kernel matrices, and the kernel trick. Cover’s theorem and the representer theorem are introduced. Then, kernel ridge regression, support vector regression, and support vector machines are studied. Online algorithms for learning in RKH spaces, such as the kernel LMS, NORMA, and the kernel APSM are discussed. The notion of multiple kernel learning is presented and a discussion on sparse modeling for nonparametric models in the context of additive models is provided. The chapter closes with a case study for text authorship identification via the use of string kernels.

Multiple Kernel Learning Method Using MRMR Criterion and Kernel Alignment

Abstract: Multiple kernel learning (MKL) is a widely used kernel learning method, but how to select kernel is lack of theoretical guidance. The performance of MKL is depend on the users’ experience, which is difficult to choose the proper kernels in practical applications. In this paper, we propose a MKL method based on minimal redundant maximal relevance criterion and kernel alignment. The main feature of this method compared to others in the literature is that the selection of kernels is considered as a feature selection issue in the Hilbert space, and can obtain a set of base kernels with the highest relevance to the target task and the minimal redundancies among themselves. Experimental results on several benchmark classification data sets show that our proposed method can enhance the performance of MKL.