scispace - formally typeset
Search or ask a question
Topic

Multiple kernel learning

About: Multiple kernel learning is a research topic. Over the lifetime, 1630 publications have been published within this topic receiving 56082 citations.


Papers
More filters
Posted ContentDOI
21 Oct 2022
TL;DR: In this article , the authors perform an empirical study of the evolution of the empirical NTK under standard and adversarial training, aiming to disambiguate the effect of adversarial learning on kernel learning and lazy training.
Abstract: Two key challenges facing modern deep learning are mitigating deep networks' vulnerability to adversarial attacks and understanding deep learning's generalization capabilities. Towards the first issue, many defense strategies have been developed, with the most common being Adversarial Training (AT). Towards the second challenge, one of the dominant theories that has emerged is the Neural Tangent Kernel (NTK) -- a characterization of neural network behavior in the infinite-width limit. In this limit, the kernel is frozen, and the underlying feature map is fixed. In finite widths, however, there is evidence that feature learning happens at the earlier stages of the training (kernel learning) before a second phase where the kernel remains fixed (lazy training). While prior work has aimed at studying adversarial vulnerability through the lens of the frozen infinite-width NTK, there is no work that studies the adversarial robustness of the empirical/finite NTK during training. In this work, we perform an empirical study of the evolution of the empirical NTK under standard and adversarial training, aiming to disambiguate the effect of adversarial training on kernel learning and lazy training. We find under adversarial training, the empirical NTK rapidly converges to a different kernel (and feature map) than standard training. This new kernel provides adversarial robustness, even when non-robust training is performed on top of it. Furthermore, we find that adversarial training on top of a fixed kernel can yield a classifier with $76.1\%$ robust accuracy under PGD attacks with $\varepsilon = 4/255$ on CIFAR-10.
Journal ArticleDOI
TL;DR: A class-pair-guided MKL method to integrate the heterogeneous features from multispectral image (MSI) and light detection and ranging (LiDAR) data that achieves the best performance in terms of classification accuracies in integrating the HFs for classification when compared with several state-of-the-art algorithms.
Abstract: . In recent years, many studies on remote sensing image classification have shown that using multiple features from different data sources can effectively improve the classification accuracy. As a very powerful means of learning, multiple kernel learning (MKL) can conveniently be embedded in a variety of characteristics. The conventional combined kernel learned by MKL can be regarded as the compromise of all basic kernels for all classes in classification. It is the best of the whole, but not optimal for each specific class. For this problem, this paper proposes a class-pair-guided MKL method to integrate the heterogeneous features (HFs) from multispectral image (MSI) and light detection and ranging (LiDAR) data. In particular, the one-against-one strategy is adopted, which converts multiclass classification problem to a plurality of two-class classification problem. Then, we select the best kernel from pre-constructed basic kernels set for each class-pair by kernel alignment (KA) in the process of classification. The advantage of the proposed method is that only the best kernel for the classification of any two classes can be retained, which leads to greatly enhanced discriminability. Experiments are conducted on two real data sets, and the experimental results show that the proposed method achieves the best performance in terms of classification accuracies in integrating the HFs for classification when compared with several state-of-the-art algorithms.
Proceedings ArticleDOI
01 May 2018
TL;DR: This paper investigates fusion of different types of classifiers for activity recognition on first-person videos in a data-driven approach and shows improvements in recognition comparison to traditional methods.
Abstract: In this paper, we investigate fusion of different types of classifiers for activity recognition on first-person videos in a data-driven approach. The algorithm first uses the classifiers, which are composed of kernel and descriptor combinations, through well-known AdaBoost trials. After all trials, classifiers are ordered and assigned ranks with respect to their performances in each trial separately. These classifiers compose a candidate list according to their performance ranks. Classifiers in the candidate list are employed together on the training set again. Classifiers in most successful candidate lists are combined as final classifiers. Our experiments show improvements in recognition comparison to traditional methods.
Book ChapterDOI
07 Oct 2020
TL;DR: In this article, a new method for classification of breast lesions in magnetic resonance imaging is proposed, which uses the pre-trained ResNet-50 architecture for extracting a set of image features that are then used by an SVM model for differentiating between positive and negative findings.
Abstract: Training of deep learning models requires large and properly labeled datasets, which make unfeasible using it for developing computer-aided diagnosis systems in medical imaging. As an alternative, transfer learning has shown to be useful to extract deep features using architectures previously trained. In this paper, a new method for classification of breast lesions in magnetic resonance imaging is proposed, which uses the pre-trained ResNet-50 architecture for extracting a set of image features that are then used by an SVM model for differentiating between positive and negative findings. We take advantage of the ResNet-50 architecture for introducing volumetric lesion information by including three consecutive slices per lesion. Filters used as feature descriptors were selected using a multiple kernel learning method, which allows identifying those filters that provide the most relevant information for the classification task. Additionally, instead of using raw filters as features, we propose to characterize it using statistical moments, which improves the classification performance. The evaluation was conducted using a set of 146 ROIs extracted from three sequences proposed for designing abbreviated breast MRI protocols (DCE, ADC, and T2-Vista). Positive findings were identified with an AUC of 82.4 using a DCE image, and 81.08 fusing features from the three sequences.
Book ChapterDOI
15 Sep 2020
TL;DR: In this article, an adaptation of classical spectrum kernels, named monotone spectrum kernels was proposed to reflect a hierarchical feature space of sub-structures of increasing complexity, and the optimal combination of those kernels through a multiple kernel learning approach produces effective and robust deep kernels.
Abstract: A recent result in the literature states that polynomial and conjunctive features can be hierarchically organized and described by different kernels of increasing expressiveness (or complexity). Additionally, the optimal combination of those kernels through a Multiple Kernel Learning approach produces effective and robust deep kernels. In this paper, we extend this approach to structured data, showing an adaptation of classical spectrum kernels, here named monotone spectrum kernels, reflecting a hierarchical feature space of sub-structures of increasing complexity. Finally, we show that (i) our kernels adaptation does not differ significantly from classical spectrum kernels, and (ii) the optimal combination achieves better results than the single spectrum kernel.

Network Information
Related Topics (5)
Convolutional neural network
74.7K papers, 2M citations
89% related
Deep learning
79.8K papers, 2.1M citations
89% related
Feature extraction
111.8K papers, 2.1M citations
87% related
Feature (computer vision)
128.2K papers, 1.7M citations
87% related
Image segmentation
79.6K papers, 1.8M citations
86% related
Performance
Metrics
No. of papers in the topic in previous years
YearPapers
202321
202244
202172
2020101
2019113
2018114