Multiple kernel learning

About: Multiple kernel learning is a research topic. Over the lifetime, 1630 publications have been published within this topic receiving 56082 citations.

An Investigation into the Effects of Multiple Kernel Combinations on Solutions Spaces in Support Vector Machines

Abstract: The use of Multiple Kernel Learning (MKL) for Support Vector Machines (SVM) in Machine Learning tasks is a growing field of study. MKL kernels expand on traditional base kernels that are used to improve performance on non-linearly separable datasets. Multiple kernels use combinations of those base kernels to develop novel kernel shapes that allow for more diversity in the generated solution spaces. Customising these kernels to the dataset is still mostly a process of trial and error. Guidelines around what combinations to implement are lacking and usually they requires domain specific knowledge and understanding of the data. Through a brute force approach, this study tests multiple datasets against a combination of base and non-weighted MKL kernels across a range of tuning hyperparameters. The goal was to determine the effect different kernels shapes have on classification accuracy and whether the resulting values are statistically different populations. A selection of 8 different datasets are chosen and trained against a binary classifier. The research will demonstrate the power for MKL to produce new and effective kernels showing the power and usefulness of this approach.

Saliency prediction with scene structural guidance

Abstract: Previous works have suggested the role of scene information in directing gaze. The structure of a scene provides global contextual information that complements local object information in saliency prediction. In this study, we explore how scene envelopes such as openness, depth, and perspective affect visual attention in natural outdoor images. To facilitate this study, an eye tracking dataset is first built with 500 natural scene images and eye tracking data with 15 subjects free-viewing the images. We make observations on scene layout properties and propose a set of scene structural features relating to visual attention. We further integrate features from deep neural networks and use the set of complementary features for saliency prediction. Our features are independent of and can work together with many computational modules, and this work demonstrates the use of Multiple kernel learning (MKL) as an example to integrate the features at low- and high-levels. Experimental results demonstrate that our model outperforms existing methods and our scene structural features can improve the performance of other saliency models in outdoor scenes.

A spatial-spectral classification strategy for very high-resolution images using region covariance descriptors and multiple kernel learning algorithms

Abstract: Extracting and modeling the spatial information content of very high resolution (VHR) images can dramatically increase the performances of urban area classification. However, extracting spatial features is a highly challenging task. During the years, several spatial feature extraction methods have been proposed, most of which are mainly designed for grayscale images. To use these methods for a multispectral image, usually, a dimensionality reduction step is required. As a result, these methods cannot optimally extract the spatial information contents of different bands of a multispectral image. To address this issue, we proposed the use of the region covariance descriptor (RCD) for spatial feature extraction from VHR images. The RCD features consider the covariance matrix of a local neighborhood of each pixel as the features. These features can model both the spatial information and the spectral relationship between bands. The RCD features lie in a Riemannian manifold, on which the common classification algorithms cannot be applied. To overcome this, we used Riemannian kernel functions. Also, we proposed a multiple kernel learning strategy for combining RCD and spectral features. The proposed strategy was evaluated for classifying a VHR image acquired over the urban area of Tehran, Iran. Furthermore, its obtained results were compared with those of ten other common spatial feature extraction methods. The results showed that the proposed classification strategy using the RCD features yielded at least 5% higher accuracies than the other feature extraction methods.

MKL-SVM-based human detection for autonomous navigation of a robot

Abstract: This paper presents a classifier trained by a multiple kernel-learning support vector machine (MKL-SVM) to detect a human in sequential images from a video stream. The developed method consists of two aspects: multiple features consisting of HOG features and HOF features suitable for moving objects, and combined nonlinear kernels for SVM. For the purpose of real time application in autonomous navigation, the SimpleMKL algorithm is implemented into the proposed MKL-SVM classifier. It is able to converge rapidly with comparable efficiency through a weighted 2-norm regularization formulation with an additional constraint on the weights. The classifier is compared with the state-of-the-art linear SVM using a dataset called TUD-Brussels, which is available on line. The results show that the proposed classifier outperforms the Linear SVM with respect to accuracy.

Fast Learning Rate of lp-MKL and its Minimax Optimality

Abstract: In this paper, we give a new sharp generalization bound of lp-MKL which is a generalized framework of multiple kernel learning (MKL) and imposes lp-mixed-norm regularization instead of l1-mixed-norm regularization. We utilize localization techniques to obtain the sharp learning rate. The bound is characterized by the decay rate of the eigenvalues of the associated kernels. A larger decay rate gives a faster convergence rate. Furthermore, we give the minimax learning rate on the ball characterized by lp-mixed-norm in the product space. Then we show that our derived learning rate of lp-MKL achieves the minimax optimal rate on the lp-mixed-norm ball.