Multiple kernel learning

About: Multiple kernel learning is a research topic. Over the lifetime, 1630 publications have been published within this topic receiving 56082 citations.

Exploiting the structure of feature spaces in kernel learning

Abstract: The problem of learning the optimal representation for a specific task recently became an important and not trivial topic in the machine learning community. In this field, deep architectures are the current gold standard among the machine learning algorithms by generating models with several levels of abstraction discovering very complicated structures in large datasets. Kernels and Deep Neural Networks (DNNs) are the principal methods to handle the representation problem in a deep manner. A DNN uses the famous back-propagation algorithm improving the state-of-the-art performance in several different real world applications, e.g. speech recognition, object detection and signal processing. Nevertheless, DNN algorithms have some drawbacks, inherited from standard neural networks, since they are theoretically not well understood. The main problems are: the complex structure of the solution, the unclear decoupling between the representation learning phase and the model generation, long training time, and the convergence to a sub-optimal solution (because of local minima and vanishing gradient). For these reasons, in this thesis, we propose new ideas to obtain an optimal representation by exploiting the kernels theory. Kernel methods have an elegant framework that decouples learning algorithms from data representations. On the other hand, kernels also have some weaknesses, for example they do not scale and they generally bring a shallow representation. In this thesis, we propose new theory and algorithms to fill this gap and make kernel learning able to generate deeper representation and to be more scalable. Considering this scenario we propose a different point of view regarding the Multiple Kernel Learning (MKL) framework, starting from the idea of a deeper kernel. An algorithm able to combine thousands of weak kernels with low computational and memory complexities is proposed. This procedure, called EasyMKL, outperforms the state-of-the-art methods combining the fragmented information in order to create an optimal kernel for the given task. Pursuing the idea to create an optimal family of weak kernels, we create a new measure for the evaluation of the kernel expressiveness, called spectral complexity. Exploiting this measure we are able to generate families of kernels with a hierarchical structure of the features by defining a new property concerning the monotonicity of the spectral complexity. We prove the quality of these weak families of kernels developing a new methodology for the Multiple Kernel Learning (MKL). Firstly we are able to create an optimal family of weak kernels by using the monotonically spectral-complex property; then we combine the optimal family of kernels by exploiting EasyMKL, obtaining a new kernel that is specific for the task; finally, we are able to generate the model by using a kernel machine. Moreover, we highlight the connection among distance metric learning, feature learning and kernel learning by proposing a method to learn the optimal family of weak kernels for a MKL algorithm in the different context in which the combination rule is the product element-wise of kernel matrices. This algorithm is able to generate the best parameters for an anisotropic RBF kernel and, therefore, a connection naturally appears among feature weighting, combinations of kernels and metric learning. Finally, the importance of the representation is also taken into account in three tasks from real world problems where we tackle different issues such as noise data, real-time application and big data

Document Image Retrieval Using Feature Combination in Kernel Space

Abstract: The paper presents application of multiple features for word based document image indexing and retrieval. A novel framework to perform Multiple Kernel Learning for indexing using the Kernel based Distance Based Hashing is proposed. The Genetic Algorithm based framework is used for optimization. Two different features representing the structural organization of word shape are defined. The optimal combination of both the features for indexing is learned by performing MKL. The retrieval results for document collection belonging to Devanagari script are presented.

Multiple Kernel Learning-Based Uncertainty Set Construction for Robust optimization

Abstract: In robust optimization (RO), a focal point is the design of uncertainty set that delineates possible realizations of uncertainty since it heavily impacts the robustness of solutions. We propose in this paper a multiple kernel learning (MKL) based support vector clustering (SVC) method for polytopic uncertainty set construction in data-driven RO. By assuming a set of candidate piecewise linear kernel functions, the MKL framework not only derives an enclosing sphere in the input space, but also automatically derives optimal coefficients of kernel functions by only solving a quadratically constrained quadratic program. The learnt sphere turns out to be a compact polyhedral uncertainty set to be used in RO, which helps reducing the conservatism of robust solutions. Meanwhile, although massive data samples and kernel functions are used in MKL, the induced polytopic uncertainty set tends to have a succinct expression, thereby well preserving the tractability of the induced optimization problem. It also allows a decisionmaker to conveniently adjust the conservatism of the data-driven uncertainty set by manipulating only one parameter, which is user-friendly in practice. Numerical case studies are carried out to demonstrate the potential advantages of the proposed method in promoting the practicability of RO techniques.

Cholesky Factorization Based Online Sequential Multiple Kernel Extreme Learning Machine Algorithm for a Cement Clinker Free Lime Content Prediction Model

Abstract: Aiming at the difficulty in real-time measuring and the long offline measurement cycle for the content of cement clinker free lime (fCaO), it is very important to build an online prediction model for fCaO content. In this work, on the basis of Cholesky factorization, the online sequential multiple kernel extreme learning machine algorithm (COS-MKELM) is proposed. The LDLT form Cholesky factorization of the matrix is introduced to avoid the large operation amount of inverse matrix calculation. In addition, the stored initial information is utilized to realize online model identification. Then, three regression datasets are used to test the performance of the COS-MKELM algorithm. Finally, an online prediction model for fCaO content is built based on COS-MKELM. Experimental results demonstrate that the fCaO content model improves the performance in terms of learning efficiency, regression accuracy, and generalization ability. In addition, the online prediction model can be corrected in real-time when the production conditions of cement clinker change.

Multiple kernel weighting based SVM for heterogeneous image recognition system

Abstract: Kernel based machine learning such as Support Vector Machines (SVMs) have proven to be powerful for many database classification problems, especially for Content Based Image Retrieval systems (CBIR). Multiple Kernel Learning (MKL) approach was recently proposed to improve kernel based classification results. MKL approach depends essentially on the used kernels and the computation of the optimal weight coefficients. However in case of heterogeneous databases, the complexity to treat and classify images provides great difficultly to define and determine optimal kernel weights. We propose in this paper an original kernel weighting method, which is intended for Multiple Kernel based SVM classification. Depending on the relevance of kernel training rates, the proposed method allows us to ensure better classification accuracy and significantly less computation time.