Showing papers in "Computational Intelligence and Neuroscience in 2008"

Theorems on Positive Data: on the Uniqueness of NMF

Abstract: We investigate the conditions for which nonnegative matrix factorization (NMF) is unique and introduce several theorems which can determine whether the decomposition is in fact unique or not. The theorems are illustrated by several examples showing the use of the theorems and their limitations. We have shown that corruption of a unique NMF matrix by additive noise leads to a noisy estimation of the noise-free unique solution. Finally, we use a stochastic view of NMF to analyze which characterization of the underlying model will result in an NMF with small estimation errors.

Probabilistic latent variable models as nonnegative factorizations.

Abstract: This paper presents a family of probabilistic latent variable models that can be used for analysis of nonnegative data. We show that there are strong ties between nonnegative matrix factorization and this family, and provide some straightforward extensions which can help in dealing with shift invariances, higher-order decompositions and sparsity constraints. We argue through these extensions that the use of this approach allows for rapid development of complex statistical models for analyzing nonnegative data.

Extended nonnegative tensor factorisation models for musical sound source separation.

Abstract: Recently, shift-invariant tensor factorisation algorithms have been proposed for the purposes of sound source separation of pitched musical instruments. However, in practice, existing algorithms require the use of log-frequency spectrograms to allow shift invariance in frequency which causes problems when attempting to resynthesise the separated sources. Further, it is difficult to impose harmonicity constraints on the recovered basis functions. This paper proposes a new additive synthesis-based approach which allows the use of linear-frequency spectrograms as well as imposing strict harmonic constraints, resulting in an improved model. Further, these additional constraints allow the addition of a source filter model to the factorisation framework, and an extended model which is capable of separating mixtures of pitched and percussive instruments simultaneously.

Nonnegative matrix factorization with Gaussian process priors

Abstract: We present a general method for including prior knowledge in a nonnegative matrix factorization (NMF), based on Gaussian process priors. We assume that the nonnegative factors in the NMF are linked by a strictly increasing function to an underlying Gaussian process specified by its covariance function. This allows us to find NMF decompositions that agree with our prior knowledge of the distribution of the factors, such as sparseness, smoothness, and symmetries. The method is demonstrated with an example from chemical shift brain imaging.

Fast nonnegative matrix factorization algorithms using projected gradient approaches for large-scale problems

Abstract: Recently, a considerable growth of interest in projected gradient (PG) methods has been observed due to their high efficiency in solving large-scale convex minimization problems subject to linear constraints. Since the minimization problems underlying nonnegative matrix factorization (NMF) of large matrices well matches this class of minimization problems, we investigate and test some recent PG methods in the context of their applicability to NMF. In particular, the paper focuses on the following modified methods: projected Landweber, Barzilai-Borwein gradient projection, projected sequential subspace optimization (PSESOP), interior-point Newton (IPN), and sequential coordinate-wise. The proposed and implemented NMF PG algorithms are compared with respect to their performance in terms of signal-to-interference ratio (SIR) and elapsed time, using a simple benchmark of mixed partially dependent nonnegative signals.

A novel design of 4-class BCI using two binary classifiers and parallel mental tasks.

Abstract: A novel 4-class single-trial brain computer interface (BCI) based on two (rather than four or more) binary linear discriminant analysis (LDA) classifiers is proposed, which is called a “parallel BCI.” Unlike other BCIs where mental tasks are executed and classified in a serial way one after another, the parallel BCI uses properly designed parallel mental tasks that are executed on both sides of the subject body simultaneously, which is the main novelty of the BCI paradigm used in our experiments. Each of the two binary classifiers only classifies the mental tasks executed on one side of the subject body, and the results of the two binary classifiers are combined to give the result of the 4-class BCI. Data was recorded in experiments with both real movement and motor imagery in 3 able-bodied subjects. Artifacts were not detected or removed. Offline analysis has shown that, in some subjects, the parallel BCI can generate a higher accuracy than a conventional 4-class BCI, although both of them have used the same feature selection and classification algorithms.

Editorial: advances in nonnegative matrix and tensor factorization

Abstract: Nonnegative matrix factorization (NMF) and its extension known as nonnegative tensor factorization (NTF) are emerging techniques that have been proposed recently. The goal of NMF/NTF is to decompose a nonnegative data matrix into a product of lower-rank nonnegative matrices or tensors (i.e., multiway arrays). An NMF approach similar to independent component analysis (ICA) or sparse component analysis (SCA) is very useful and promising for decomposing high-dimensional datasets into a lower-dimensional space. A great deal of interest has been given very recently to NMF models and techniques due to their capability of providing new insights and relevant information on the complex latent relationships in experimental datasets, and due to providing meaningful components with physical or physiological interpretations. For example, in bioinformatics, NMF and its extensions have been successfully applied to gene expression, sequence analysis, functional characterization of genes, clustering, and text mining. The main difference between NMF and other classical factorizations such as PCA, SCA, or ICA methods relies on the nonnegativity, and usually also additional constraints such as sparseness, smoothness, and/or orthogonality imposed on the models. These constraints tend to lead to a parts-based representation of the data, because they allow only additive, not subtractive, combinations of data items. In this way, the nonnegative components or factors produced by this approach can be interpreted as parts of the data. In other words, NMF yields nonnegative factors, which can be advantageous from the point of view of interpretability of the estimated components. Furthermore, in many real applications, data have a multiway (multiway array or tensor) structure. Exemplary data are video stream (rows, columns, RGB color coordinates, time), EEG in neuroscience (channels, frequency, time, samples, conditions, subjects), bibliographic text data (keywords, papers, authors, journals), and so on. Conventional methods preprocess multiway data, arranging them into a matrix. Recently, there has been a great deal of research on multiway analysis which conserves the original multiway structure of the data. The techniques have been shown to be very useful in a number of applications, such as signal separation, feature extraction, audio coding, speech classification, image compression, spectral clustering, neuroscience, and biomedical signal analysis. This special issue focuses on the most recent advances in NMF/NTF methods, with emphasis on the efforts made particularly by the researchers from the signal processing and neuroscience area. It reports novel theoretical results, efficient algorithms, and their applications. It also provides insight into current challenging areas, and identifies future research directions. This issue includes several important contributions which cover a wide range of approaches and techniques for NMF/NTF and their applications. These contributions are summarized as follows. The first paper, entitled “Probabilistic latent variable models as nonnegative factorizations” by M. Shashanka et al., presents a family of probabilistic latent variable models that can be used for analysis of nonnegative data. The paper shows that there are strong ties between NMF and this family, and provides some straightforward extensions which can help in dealing with shift invariances, higher-order decompositions, and sparsity constraints. Furthermore, it argues through these extensions that the use of this approach allows for rapid development of complex statistical models for analyzing nonnegative data. The second paper, entitled “Fast nonnegative matrix factorization algorithms using projected gradient approaches for large-scale problems” by R. Zdunek and A. Cichocki, investigates the applicability of projected gradient (PG) methods to NMF, based on the observation that the PG methods have high efficiency in solving large-scale convex minimization problems subject to linear constraints, since the minimization problems underlying NMF of large matrices well match this class of minimization problems. In particular, the paper has investigated several modified and adopted methods, including projected Landweber method, Barzilai-Borwein gradient projection, projected sequential subspace optimization, interior-point Newton algorithm, and sequential coordinatewise minimization algorithm, and compared their performance in terms of signal-to-interference ratio and elapsed time, using a simple benchmark of mixed partially dependent nonnegative signals. The third paper, entitled “Theorems on positive data: on the uniqueness of NMF” by H. Laurberg et al., investigates the conditions for which NMF is unique, and introduces several theorems which can determine whether the decomposition is in fact unique or not. Several examples are provided to show the use of the theorems and their limitations. The paper also shows that corruption of a unique NMF matrix by additive noise leads to a noisy estimation of the noise-free unique solution. Moreover, it uses a stochastic view of NMF to analyze which characterization of the underlying model will result in an NMF with small estimation errors. The fourth paper, entitled “Nonnegative matrix factorization with Gaussian process priors” by M. N. Schmidt and H. Laurberg, presents a general method for including prior knowledge in NMF, based on Gaussian process priors. It assumes that the nonnegative factors in the NMF are linked by a strictly increasing function to an underlying Gaussian process specified by its covariance function. The NMF decompositions are found to be in agreement with the prior knowledge of the distribution of the factors, such as sparseness, smoothness, and symmetries. The fifth paper, entitled “Extended nonnegative tensor factorisation models for musical sound source separation” by D. FitzGerald et al., presents a new additive synthesis-based NTF approach which allows the use of linear-frequency spectrograms as well as imposing strict harmonic constraints, resulting in an improved model as compared with some existing shift-invariant tensor factorization algorithms in which the use of log-frequency spectrograms to allow shift invariance in frequency causes problems when attempting to resynthesize the separated sources. The paper further studies the addition of a source filter model to the factorization framework, and presents an extended model which is capable of separating mixtures of pitched and percussive instruments simultaneously. The sixth paper, entitled “Gene tree labeling using nonnegative matrix factorization on biomedical literature” by K. E. Heinrich et al., addresses a challenging problem for biological applications, that is, identifying functional groups of genes. It examines the NMF technique for labeling hierarchical trees. It proposes a generic labeling algorithm as well as an evaluation technique, and discusses the effects of different NMF parameters with regard to convergence and labeling accuracy. The primary goals of this paper are to provide a qualitative assessment of the NMF and its various parameters and initialization, to provide an automated way to classify biomedical data, and to provide a method for evaluating labeled data assuming a static input tree. This paper also proposes a method for generating gold standard trees. The seventh paper, entitled “Single-trial decoding of bistable perception based on sparse nonnegative tensor decomposition” by Z. Wang et al., presents a sparse NTF-based method to extract features from the local field potential (LFP), collected from the middle temporal visual cortex in a macaque monkey, for decoding its bistable structure-from-motion perception. The advantages of the sparse NTF-based feature-extraction approach lie in its capability to yield components common across the space, time, and frequency domains, yet discriminative across different conditions without prior knowledge of the discriminating frequency bands and temporal windows for a specific subject. The results suggest that imposing the sparseness constraints on the NTF improves extraction of the gamma band feature which carries the most discriminative information for bistable perception. The eighth paper, entitled “Pattern expression nonnegative matrix factorization: algorithm and applications to blind source separation” by J. Zhang et al., presents a pattern expression NMF (PE-NMF) approach from the view point of using basis vectors most effectively to express patterns. Two regularization or penalty terms are introduced to be added to the original loss function of a standard NMF for effective expression of patterns with basis vectors in the PE-NMF. A learning algorithm is presented, and the convergence of the algorithm is proved theoretically. Three illustrative examples for blind source separation including heterogeneity correction for gene microarray data indicate that the sources can be successfully recovered with the proposed PE-NMF when the two parameters can be suitably chosen from prior knowledge of the problem. The last paper, entitled “Robust object recognition under partial occlusions using NMF” by D. Soukup and I. Bajla, studies NMF methods for recognition tasks with occluded objects. The paper analyzes the influence of sparseness on recognition rates for various dimensions of subspaces generated for two image databases, ORL face database, and USPS handwritten digit database. It also studies the behavior of four types of distances between a projected unknown image object and feature vectors in NMF subspaces generated for training data. In the recognition phase, partial occlusions in the test images have been modeled by putting two randomly large, randomly positioned black rectangles into each test image.

Parametric and nonparametric EEG analysis for the evaluation of EEG activity in young children with controlled epilepsy

Abstract: There is an important evidence of differences in the EEG frequency spectrum of control subjects as compared to epileptic subjects. In particular, the study of children presents difficulties due to the early stages of brain development and the various forms of epilepsy indications. In this study, we consider children that developed epileptic crises in the past but without any other clinical, psychological, or visible neurophysiological findings. The aim of the paper is to develop reliable techniques for testing if such controlled epilepsy induces related spectral differences in the EEG. Spectral features extracted by using nonparametric, signal representation techniques (Fourier and wavelet transform) and a parametric, signal modeling technique (ARMA) are compared and their effect on the classification of the two groups is analyzed. The subjects performed two different tasks: a control (rest) task and a relatively difficult math task. The results show that spectral features extracted by modeling the EEG signals recorded from individual channels by an ARMA model give a higher discrimination between the two subject groups for the control task, where classification scores of up to 100% were obtained with a linear discriminant classifier.

Pattern expression nonnegative matrix factorization: algorithm and applications to blind source separation.

Abstract: Independent component analysis (ICA) is a widely applicable and effective approach in blind source separation (BSS), with limitations that sources are statistically independent. However, more common situation is blind source separation for nonnegative linear model (NNLM) where the observations are nonnegative linear combinations of nonnegative sources, and the sources may be statistically dependent. We propose a pattern expression nonnegative matrix factorization (PE-NMF) approach from the view point of using basis vectors most effectively to express patterns. Two regularization or penalty terms are introduced to be added to the original loss function of a standard nonnegative matrix factorization (NMF) for effective expression of patterns with basis vectors in the PE-NMF. Learning algorithm is presented, and the convergence of the algorithm is proved theoretically. Three illustrative examples on blind source separation including heterogeneity correction for gene microarray data indicate that the sources can be successfully recovered with the proposed PE-NMF when the two parameters can be suitably chosen from prior knowledge of the problem.

Robust object recognition under partial occlusions using NMF.

Abstract: In recent years, nonnegative matrix factorization (NMF) methods of a reduced image data representation attracted the attention of computer vision community These methods are considered as a convenient part-based representation of image data for recognition tasks with occluded objects A novel modification in NMF recognition tasks is proposed which utilizes the matrix sparseness control introduced by Hoyer We have analyzed the influence of sparseness on recognition rates (RRs) for various dimensions of subspaces generated for two image databases, ORL face database, and USPS handwritten digit database We have studied the behavior of four types of distances between a projected unknown image object and feature vectors in NMF subspaces generated for training data One of these metrics also is a novelty we proposed In the recognition phase, partial occlusions in the test images have been modeled by putting two randomly large, randomly positioned black rectangles into each test image

Gene tree labeling using nonnegative matrix factorization on biomedical literature

Abstract: Identifying functional groups of genes is a challenging problem for biological applications. Text mining approaches can be used to build hierarchical clusters or trees from the information in the biological literature. In particular, the nonnegative matrix factorization (NMF) is examined as one approach to label hierarchical trees. A generic labeling algorithm as well as an evaluation technique is proposed, and the effects of different NMF parameters with regard to convergence and labeling accuracy are discussed. The primary goals of this study are to provide a qualitative assessment of the NMF and its various parameters and initialization, to provide an automated way to classify biomedical data, and to provide a method for evaluating labeled data assuming a static input tree. As a byproduct, a method for generating gold standard trees is proposed.

Single-trial decoding of bistable perception based on sparse nonnegative tensor decomposition

Abstract: The study of the neuronal correlates of the spontaneous alternation in perception elicited by bistable visual stimuli is promising for understanding the mechanism of neural information processing and the neural basis of visual perception and perceptual decision-making. In this paper, we develop a sparse nonnegative tensor factorization-(NTF)-based method to extract features from the local field potential (LFP), collected from the middle temporal (MT) visual cortex in a macaque monkey, for decoding its bistable structure-from-motion (SFM) perception. We apply the feature extraction approach to the multichannel time-frequency representation of the intracortical LFP data. The advantages of the sparse NTF-based feature extraction approach lies in its capability to yield components common across the space, time, and frequency domains yet discriminative across different conditions without prior knowledge of the discriminating frequency bands and temporal windows for a specific subject. We employ the support vector machines (SVMs) classifier based on the features of the NTF components for single-trial decoding the reported perception. Our results suggest that although other bands also have certain discriminability, the gamma band feature carries the most discriminative information for bistable perception, and that imposing the sparseness constraints on the nonnegative tensor factorization improves extraction of this feature.

Performance of a Self-Paced Brain Computer Interface on Data Contaminated with Eye-Movement Artifacts and on Data Recorded in a Subsequent Session

Abstract: The performance of a specific self-paced BCI (SBCI) is investigated using two different datasets to determine its suitability for using online: (1) data contaminated with large-amplitude eye movements, and (2) data recorded in a session subsequent to the original sessions used to design the system. No part of the data was rejected in the subsequent session. Therefore, this dataset can be regarded as a “pseudo-online” test set. The SBCI under investigation uses features extracted from three specific neurological phenomena. Each of these neurological phenomena belongs to a different frequency band. Since many prominent artifacts are either of mostly low-frequency (e.g., eye movements) or mostly high-frequency nature (e.g., muscle movements), it is expected that the system shows a fairly robust performance over artifact-contaminated data. Analysis of the data of four participants using epochs contaminated with large-amplitude eye-movement artifacts shows that the system's performance deteriorates only slightly. Furthermore, the system's performance during the session subsequent to the original sessions remained largely the same as in the original sessions for three out of the four participants. This moderate drop in performance can be considered tolerable, since allowing artifact-contaminated data to be used as inputs makes the system available for users at ALL times.

Asymmetric variate generation via a parameterless dual neural learning algorithm

Abstract: In a previous work (S. Fiori, 2006), we proposed a random number generator based on a tunable non-linear neural system, whose learning rule is designed on the basis of a cardinal equation from statistics and whose implementation is based on look-up tables (LUTs). The aim of the present manuscript is to improve the above-mentioned random number generation method by changing the learning principle, while retaining the efficient LUT-based implementation. The new method proposed here proves easier to implement and relaxes some previous limitations.