Multiple kernel learning

About: Multiple kernel learning is a research topic. Over the lifetime, 1630 publications have been published within this topic receiving 56082 citations.

Multiple Kernel Learning: A Unifying Probabilistic Viewpoint

Abstract: Wepresentaprobabilisticviewpointtomultiplekernellearningunifyingwell-knownregu-larisedriskapproachesandrecentadvancesinapproximateBayesianinferencerelaxations.Theframeworkproposesageneralobjectivefunctionsuitableforregression,robustregres-sion and classification that is lower bound of the marginal likelihood and contains manyregularisedriskapproachesasspecialcases. Furthermore,wederiveanefficientandprov-ablyconvergentoptimisationalgorithm.Keywords: Multiple kernel learning, approximate Bayesian inference, double loop algo-rithms,Gaussianprocesses 1. Introduction Nonparametric kernel methods, cornerstones of machine learning today, can be seen fromdifferentangles: asregularisedriskminimisationinfunctionspaces(ScholkopfandSmola,2002), or as probabilistic Gaussian process methods (Rasmussen and Williams,2006). Inthesetechniques,thekernel(orequivalentlycovariance)functionencodesinterpolationchar-acteristics from observed to unseen points, and two basic statistical problems have to bemastered. First,alatentfunctionmustbepredictedwhichfitsdatawell,yetisassmoothaspossiblegiventhefixedkernel. Second,thekernelfunctionparametershavetobelearnedaswell,tobestsupportpredictionswhichareofprimaryinterest. Whilethefirstproblemissimplerandhasbeenaddressedmuchmorefrequentlysofar,thecentralroleoflearningthecovariancefunctioniswellacknowledged, andasubstantialnumberofmethodsfor“learn-ing the kernel”, “multiple kernel learning”, or “evidence maximisation” are available now.However, much of this work has firmly been associated with one of the “camps” (referredto as regularised risk and probabilistic in the sequel) with surprisingly little crosstalk oracknowledgmentsofpriorworkacrossthisboundary. Inthispaper,weclarifytherelation-ship between major regularised risk and probabilistic kernel learning techniques precisely,pointingoutadvantagesandpitfallsofeither,aswellasalgorithmicsimilaritiesleadingtonovelpowerfulalgorithms.Wedevelopacommonanalyticalandalgorithmicalframeworkencompassingapproachesfrombothcampsandprovideclearinsightsintotheoptimisationstructure. Eventhough,most of the optimisation is non convex, we show how to operate a provably convergent“almostNewton”methodnevertheless. Eachstepisnotmuchmoreexpensivethanagradient

RISK: A random optimization interactive system based on kernel learning for predicting breast cancer disease progression

Abstract: Evaluating disease progression risk is a key issue in medicine that has been revolutionized by the advent of machine learning approaches and the wide availability of medical data in electronic form. It is time to provide physicians with near-to-the-clinical-practice and effective tools to spread this important technological innovation. In this paper, we describe RISK, a web service that implements a multiple kernel learning approach for predicting breast cancer disease progression. We report on the experience of the BIBIOFAR project where RISK Web Predictor has been developed and tested. Results of our system demonstrate that this kind of approaches can effectively support physicians in the evaluation of risk.

On the Impact of Regularization Variation on Localized Multiple Kernel Learning

Abstract: This brief analyzes the effects of regularization variations in the localized kernel weights on the hypothesis generated by localized multiple kernel learning (LMKL) algorithms. Recent research on LMKL includes imposing different regularizations on the localized kernel weights and has led to varying formulations and solution strategies. Following the stability analysis theory as presented by Bousquet and Elisseeff, we give stability bounds based on the norm of the variation of localized kernel weights for three LMKL methods cast in the support vector machine classification framework, including vector $\ell _{p}$ -norm LMKL, matrix-regularized $(r,p)$ -norm LMKL, and samplewise $\ell _{p}$ -norm LMKL. Further comparison of these bounds helps to qualitatively reveal the performance differences produced by these regularization methods, that is, matrix-regularized LMKL achieves superior performance, followed by vector $\ell _{p}$ -norm LMKL and samplewise $\ell _{p}$ -norm LMKL. Finally, a set of experimental results on ten benchmark machine learning UCI data sets is reported and shown to empirically support our theoretical analysis.

Ship Target Discrimination in SAR Images Based on BOW Model With Multiple Features and Spatial Pyramid Matching

Abstract: To eliminate the false alarms in the ship target detection effectively for synthetic aperture radar (SAR) images in complex scenes, this article present a novel ship target discrimination algorithm based on bag of words (BOW) model with multiple features and spatial pyramid matching (SPM), which is named MF-SPM-BOW. The proposed discrimination method mainly contains three stages. First, the SAR scale-invariant feature transform (SAR-SIFT) descriptors and gray-level co-occurrence matrix (GLCM) descriptors are extracted as local features to describe the gradient information and texture information of local regions of an image chip. Then, the SPM technique considering its spatial location information-keeping capability is employed to generate global features with excellent discrimination ability. Finally, the support vector machine (SVM) discriminator based on multiple kernel learning is applied to realize feature fusion in image layer and thus identify targets and clutter. Experimental results show that compared with the traditional discrimination methods and the BOW model discrimination methods, the proposed SAR ship target discrimination algorithm achieves better discrimination performance, which can eliminate most of the false alarms in candidate ship target chips effectively.

A multitask multiple kernel learning formulation for discriminating early- and late-stage cancers

Abstract: Motivation Genomic information is increasingly being used in diagnosis, prognosis and treatment of cancer. The severity of the disease is usually measured by the tumor stage. Therefore, identifying pathways playing an important role in progression of the disease stage is of great interest. Given that there are similarities in the underlying mechanisms of different cancers, in addition to the considerable correlation in the genomic data, there is a need for machine learning methods that can take these aspects of genomic data into account. Furthermore, using machine learning for studying multiple cancer cohorts together with a collection of molecular pathways creates an opportunity for knowledge extraction. Results We studied the problem of discriminating early- and late-stage tumors of several cancers using genomic information while enforcing interpretability on the solutions. To this end, we developed a multitask multiple kernel learning (MTMKL) method with a co-clustering step based on a cutting-plane algorithm to identify the relationships between the input tasks and kernels. We tested our algorithm on 15 cancer cohorts and observed that, in most cases, MTMKL outperforms other algorithms (including random forests, support vector machine and single-task multiple kernel learning) in terms of predictive power. Using the aggregate results from multiple replications, we also derived similarity matrices between cancer cohorts, which are, in many cases, in agreement with available relationships reported in the relevant literature. Availability and implementation Our implementations of support vector machine and multiple kernel learning algorithms in R are available at https://github.com/arezourahimi/mtgsbc together with the scripts that replicate the reported experiments. Supplementary information Supplementary data are available at Bioinformatics online.