Showing papers by "Klaus-Robert Müller published in 2002"

Constructing boosting algorithms from SVMs: an application to one-class classification

Abstract: We show via an equivalence of mathematical programs that a support vector (SV) algorithm can be translated into an equivalent boosting-like algorithm and vice versa. We exemplify this translation procedure for a new algorithm: one-class leveraging, starting from the one-class support vector machine (1-SVM). This is a first step toward unsupervised learning in a boosting framework. Building on so-called barrier methods known from the theory of constrained optimization, it returns a function, written as a convex combination of base hypotheses, that characterizes whether a given test point is likely to have been generated from the distribution underlying the training data. Simulations on one-class classification problems demonstrate the usefulness of our approach.

A resampling approach to estimate the stability of one-dimensional or multidimensional independent components

Abstract: When applying unsupervised learning techniques in biomedical data analysis, a key question is whether the estimated parameters of the studied system are reliable. In other words, can we assess the quality of the result produced by our learning technique? We propose resampling methods to tackle this question and illustrate their usefulness for blind-source separation (BSS). We demonstrate that our proposed reliability estimation can be used to discover stable one-dimensional or multidimensional independent components, to choose the appropriate BSS-model, to enhance significantly the separation performance, and, most importantly, to flag components that carry physical meaning. Application to different biomedical testbed data sets (magnetoencephalography (MEG)/electrocardiography (ECG)-recordings) underline the usefulness of our approach.

Combining Features for BCI

Abstract: Recently, interest is growing to develop an effective communication interface connecting the human brain to a computer, the 'Brain-Computer Interface' (BCI). One motivation of BCI research is to provide a new communication channel substituting normal motor output in patients with severe neuromuscular disabilities. In the last decade, various neuro-physiological cortical processes, such as slow potential shifts, movement related potentials (MRPs) or event-related desynchronization (ERD) of spontaneous EEG rhythms, were shown to be suitable for BCI, and, consequently, different independent approaches of extracting BCI-relevant EEG-features for single-trial analysis are under investigation. Here, we present and systematically compare several concepts for combining such EEG-features to improve the single-trial classification. Feature combinations are evaluated on movement imagination experiments with 3 subjects where EEG-features are based on either MRPs or ERD, or both. Those combination methods that incorporate the assumption that the single EEG-features are physiologically mutually independent outperform the plain method of 'adding' evidence where the single-feature vectors are simply concatenated. These results strengthen the hypothesis that MRP and ERD reflect at least partially independent aspects of cortical processes and open a new perspective to boost BCI effectiveness.

Going Metric: Denoising Pairwise Data

Abstract: Pairwise data in empirical sciences typically violate metricity, either due to noise or due to fallible estimates, and therefore are hard to analyze by conventional machine learning technology. In this paper we therefore study ways to work around this problem. First, we present an alternative embedding to multi-dimensional scaling (MDS) that allows us to apply a variety of classical machine learning and signal processing algorithms. The class of pair-wise grouping algorithms which share the shift-in variance property is statistically invariant under this embedding procedure, leading to identical assignments of objects to clusters. Based on this new vectorial representation, denoising methods are applied in a second step. Both steps provide a theoretically well controlled setup to translate from pairwise data to the respective denoised metric representation. We demonstrate the practical usefulness of our theoretical reasoning by discovering structure in protein sequence data bases, visibly improving performance upon existing automatic methods.

New Methods for Splice Site Recognition

Abstract: Splice sites are locations in DNA which separate protein-coding regions (exons) from noncoding regions (introns). Accurate splice site detectors thus form important components of computational gene finders. We pose splice site recognition as a classification problem with the classifier learnt from a labeled data set consisting of only local information around the potential splice site. Note that finding the correct position of splice sites without using global information is a rather hard task. We analyze the genomes of the nematode Caenorhabditis elegans and of humans using specially designed support vector kernels. One of the kernels is adapted from our previous work on detecting translation initiation sites in vertebrates and another uses an extension to the well-known Fisher-kernel. We find excellent performance on both data sets.

Clustering with the Fisher Score

Abstract: Recently the Fisher score (or the Fisher kernel) is increasingly used as a feature extractor for classification problems The Fisher score is a vector of parameter derivatives of loglikelihood of a probabilistic model This paper gives a theoretical analysis about how class information is preserved in the space of the Fisher score, which turns out that the Fisher score consists of a few important dimensions with class information and many nuisance dimensions When we perform clustering with the Fisher score, K-Means type methods are obviously inappropriate because they make use of all dimensions So we will develop a novel but simple clustering algorithm specialized for the Fisher score, which can exploit important dimensions This algorithm is successfully tested in experiments with artificial data and real data (amino acid sequences)

Selecting ridge parameters in infinite dimensional hypothesis spaces

Abstract: Previously, an unbiased estimator of the generalization error called the subspace information criterion (SIC) was proposed for a finite dimensional reproducing kernel Hilbert space (RKHS). In this paper, we extend SIC so that it can be applied to any RKHSs including infinite dimensional ones. Computer simulations show that the extended SIC works well in ridge parameter selection.

Selecting Ridge Parameters in Infinite Dimensional Hypothesis Spaces

Abstract: Previously, an unbiased estimator of the generalization error called the subspace information criterion (SIC) was proposed for a finite dimensional reproducing kernel Hilbert space (RKHS). In this paper, we extend SIC so that it can be applied to any RKHSs including infinite dimensional ones. Computer simulations show that the extended SIC works well in ridge parameter selection.