Showing papers by "Tulay Adali published in 2015"

Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects

Abstract: In various disciplines, information about the same phenomenon can be acquired from different types of detectors, at different conditions, in multiple experiments or subjects, among others. We use the term “modality” for each such acquisition framework. Due to the rich characteristics of natural phenomena, it is rare that a single modality provides complete knowledge of the phenomenon of interest. The increasing availability of several modalities reporting on the same system introduces new degrees of freedom, which raise questions beyond those related to exploiting each modality separately. As we argue, many of these questions, or “challenges,” are common to multiple domains. This paper deals with two key issues: “why we need data fusion” and “how we perform it.” The first issue is motivated by numerous examples in science and technology, followed by a mathematical framework that showcases some of the benefits that data fusion provides. In order to address the second issue, “diversity” is introduced as a key concept, and a number of data-driven solutions based on matrix and tensor decompositions are discussed, emphasizing how they account for diversity across the data sets. The aim of this paper is to provide the reader, regardless of his or her community of origin, with a taste of the vastness of the field, the prospects, and the opportunities that it holds.

Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects This paper provides an overview of the main challenges in multimodal data fusion acrossvariousdisciplinesandaddressestwokeyissues:''whyweneeddatafusion''and ''how we perform it.''

Abstract: In various disciplines, information about the same phenomenon can be acquired from different types of detectors, at different conditions, in multiple experiments or subjects, among others. We use the term ''modality'' for each such acquisition framework.Duetothe richcharacteristics of natural phenomena, it is rare that a single modality provides complete knowledge of the phenomenon of interest. The increasing availability of several modalities reporting on the same system introduces new degrees of freedom, which raise questions beyond those related to exploiting each modality separately. As we argue, many of these questions, or ''challenges,'' are common to multiple domains. This paper deals with two key issues: ''why we need data fusion'' and ''how we perform it.'' The first issue is motivated by numerous examples in science and technology, followed by a mathematical framework that showcases some of the benefits that data fusion provides. In order to address the second issue, ''diversity'' is introduced as a key concept, and a number of data-driven solutions based on matrix and tensor decompositions are discussed, emphasizing how they account for diversity across the data sets. The aim of this paper is to provide the reader, regardless of his or her community of origin, with a taste of the vastness of the field, the prospects, and the opportunities that it holds.

Linked Component Analysis from Matrices to High Order Tensors: Applications to Biomedical Data

Abstract: With the increasing availability of various sensor technologies, we now have access to large amounts of multi-block (also called multi-set, multi-relational, or multi-view) data that need to be jointly analyzed to explore their latent connections. Various component analysis methods have played an increasingly important role for the analysis of such coupled data. In this paper, we first provide a brief review of existing matrix-based (two-way) component analysis methods for the joint analysis of such data with a focus on biomedical applications. Then, we discuss their important extensions and generalization to multi-block multiway (tensor) data. We show how constrained multi-block tensor decomposition methods are able to extract similar or statistically dependent common features that are shared by all blocks, by incorporating the multiway nature of data. Special emphasis is given to the flexible common and individual feature analysis of multi-block data with the aim to simultaneously extract common and individual latent components with desired properties and types of diversity. Illustrative examples are given to demonstrate their effectiveness for biomedical data analysis.

Multimodal Data Fusion Using Source Separation: Application to Medical Imaging

Abstract: The joint independent component analysis (jICA) and the transposed independent vector analysis (tIVA) models are two effective solutions based on blind source separation (BSS) that enable fusion of data from multiple modalities in a symmetric and fully multivariate manner The previous paper in this special issue discusses the properties and the main issues in the implementation of these two models In this accompanying paper, we consider the application of these two models to fusion of multimodal medical imaging data—functional magnetic resonance imaging (fMRI), structural MRI (sMRI), and electroencephalography (EEG) data collected from a group of healthy controls and patients with schizophrenia performing an auditory oddball task We show how both models can be used to identify a set of components that report on differences between the two groups, jointly , for all the modalities used in the study We discuss the importance of algorithm and order selection as well as tradeoffs involved in the selection of one model over another We note that for the selected data set, especially given the limited number of subjects available for the study, jICA provides a more desirable solution, however the use of an ICA algorithm that uses flexible density matching provides advantages over the most widely used algorithm, Infomax, for the problem

Multimodal Data Fusion Using Source Separation: Two Effective Models Based on ICA and IVA and Their Properties

Abstract: Fusion of information from multiple sets of data in order to extract a set of features that are most useful and relevant for the given task is inherent to many problems we deal with today. Since, usually, very little is known about the actual interaction among the data sets, it is highly desirable to minimize the underlying assumptions. This has been the main reason for the growing importance of data-driven methods, and in particular of independent component analysis (ICA) as it provides useful decompositions with a simple generative model and using only the assumption of statistical independence. A recent extension of ICA, independent vector analysis (IVA), generalizes ICA to multiple data sets by exploiting the statistical dependence across the data sets, and hence, as we discuss in this paper, provides an attractive solution to fusion of data from multiple data sets along with ICA. In this paper, we focus on two multivariate solutions for multimodal data fusion that let multiple modalities fully interact for the estimation of underlying features that jointly report on all modalities. One solution is the joint ICA model that has found wide application in medical imaging, and the second one is the transposed IVA model introduced here as a generalization of an approach based on multiset canonical correlation analysis. In the discussion, we emphasize the role of diversity in the decompositions achieved by these two models, and present their properties and implementation details to enable the user make informed decisions on the selection of a model along with its associated parameters. Discussions are supported by simulation results to help highlight the main issues in the implementation of these methods.

Multimodal Data Fusion Using Source Separation: Two Effective Models Based on ICA and IVA and Their Properties This paper introduces two powerful data-driven models and provides guidance on the selection of a given model and its implementation while emphasizing the general applicability of the two models.

Abstract: Fusion of information from multiple sets of data in order to extract a set of features that are most useful and re- levant for the given task is inherent to many problems we deal with today. Since, usually, very little is known about the actual interaction among the data sets, it is highly desirable to mini- mize the underlying assumptions. This has been the main rea- son for the growing importance of data-driven methods, and in particular of independent component analysis (ICA) as it pro- vides useful decompositions with a simple generative model and using only the assumption of statistical independence. A recent extension of ICA, independent vector analysis (IVA), generalizes ICA to multiple data sets by exploiting the statis- tical dependence across the data sets, and hence, as we discuss in this paper, provides an attractive solution to fusion of data from multiple data sets along with ICA. In this paper, we focus on two multivariate solutions for multimodal data fusion that let multiple modalities fully interact for the estimation of underlying features that jointly report on all modalities. One solution is the joint ICA model that has found wide application in medical imaging, and the second one is the transposed IVA model introduced here as a generalization of an approach based on multiset canonical correlation analysis. In the discus- sion, we emphasize the role of diversity in the decompositions achieved by these two models, and present their properties and implementation details to enable the user make informed decisions on the selection of a model along with its associated parameters. Discussions are supported by simulation results to help highlight the main issues in the implementation of these methods.

Multimodal Data Fusion Using Source Separation: Application to Medical Imaging This paper demonstrates the application of the two models introduced in the previous paper to fusion of medical imaging data from three modalities: functional magnetic resonance imaging (MRI), structural MRI, and electroencephalography data and discusses the tradeoffs in various modeling and parameter choices.

Abstract: The joint independent component analysis (jICA) thatenablefusionofdatafrommultiplemodalitiesinasymmetric and fully multivariate manner. The previous paper in this special issue discusses the properties and the main issues in the implementation of these two models. In this accompanying paper, we consider the application of these two models to fusion of multimodal medical imaging dataVfunctional magnetic reso- nance imaging (fMRI), structural MRI (sMRI), and electroenceph- alography (EEG) data collected from a group of healthy controls and patients with schizophrenia performing an auditory oddball task. We show how both models can be used to identify a set of components that report on differences between the two groups, jointly, for all the modalities used in the study. We discuss the importance of algorithm and order selection as well as tradeoffs involved in the selection of one model over another. We note that for the selected data set, especially given the limited number of subjects available for the study, jICA provides a more desirable solution, however the use of an ICA algorithm that uses flexible density matching provides advantages over the most widely used algorithm, Infomax, for the problem.

Independent Vector Analysis for Gradient Artifact Removal in Concurrent EEG-fMRI Data

Abstract: We consider the problem of removing gradient artifact from electroencephalogram (EEG) signal, recorded concurrently with functional magnetic resonance imaging (fMRI) acquisition. We estimate the artifact by exploiting its quasi-periodicity over the epochs and its similarity over the different channels by using independent vector analysis, a recent extension of independent component analysis for multiple datasets. The method fully makes use of the spatio-temporal information by using spatial dependences across channels to estimate the artifact for a particular channel. Thus, it provides robustness with respect to uncontrollable changes such as head movement and fluctuations in the $B_0$ field during the acquisition. Results using both simulated data with gradient artifact and EEG data collected concurrently with fMRI show the desirable performance of the new method.

Spatial Variance in Resting fMRI Networks of Schizophrenia Patients: An Independent Vector Analysis

Abstract: Spatial variability in resting functional MRI (fMRI) brain networks has not been well studied in schizophrenia, a disease known for both neurodevelopmental and widespread anatomic changes. Motivated by abundant evidence of neuroanatomical variability from previous studies of schizophrenia, we draw upon a relatively new approach called independent vector analysis (IVA) to assess this variability in resting fMRI networks. IVA is a blind-source separation algorithm, which segregates fMRI data into temporally coherent but spatially independent networks and has been shown to be especially good at capturing spatial variability among subjects in the extracted networks. We introduce several new ways to quantify differences in variability of IVA-derived networks between schizophrenia patients (SZs = 82) and healthy controls (HCs = 89). Voxelwise amplitude analyses showed significant group differences in the spatial maps of auditory cortex, the basal ganglia, the sensorimotor network, and visual cortex. Tests for differences (HC-SZ) in the spatial variability maps suggest, that at rest, SZs exhibit more activity within externally focused sensory and integrative network and less activity in the default mode network thought to be related to internal reflection. Additionally, tests for difference of variance between groups further emphasize that SZs exhibit greater network variability. These results, consistent with our prediction of increased spatial variability within SZs, enhance our understanding of the disease and suggest that it is not just the amplitude of connectivity that is different in schizophrenia, but also the consistency in spatial connectivity patterns across subjects.

An efficient multivariate generalized Gaussian distribution estimator: Application to IVA

Abstract: Due to its simple parametric form, multivariate generalized Gaussian distribution (MGGD) has been widely used for modeling vector-valued signals. Therefore, efficient estimation of its parameters is of significant interest for a number of applications. Independent vector analysis (IVA) is a generalization of independent component analysis (ICA) that makes full use of the statistical dependence across multiple datasets to achieve source separation, and can take both second and higher-order statistics into account. MGGD provides an effective model for IVA as well as for modeling the latent multivariate variables-sources-and the performance of the IVA algorithm highly depends on the estimation of the source parameters. In this paper, we propose an efficient estimation technique based on the Fisher scoring (FS) and demonstrate its successful application to IVA. We quantify the performance of MGGD parameter estimation using FS and further verify the effectiveness of the new IVA algorithm using simulations.

General Nonunitary Constrained ICA and its Application to Complex-Valued fMRI Data

Abstract: Constrained independent component analysis (C-ICA) algorithms provide an effective way to introduce prior information into the complex- and real-valued ICA framework. The work in this area has focus on adding constraints to the objective function of algorithms that assume a unitary demixing matrix. The unitary condition is required in order to decouple-isolate-the constraints applied for each individual source. This assumption limits the optimization space and, therefore, the separation performance of C-ICA algorithms. We generalize the existing C-ICA framework by using a novel decoupling method that preserves the larger optimization space for the demixing matrix. This framework allows for the constraining of either the sources or the mixing coefficients. A constrained version of the nonunitary entropy bound minimization algorithm is introduced and applied to actual complex-valued fMRI data. We show that constraining the mixing parameters using a temporal constraint improves the estimation of the spatial map and timecourses of task-related components.

A New Riemannian Averaged Fixed-Point Algorithm for MGGD Parameter Estimation

Abstract: Multivariate generalized Gaussian distribution (MGGD) has been an attractive solution to many signal processing problems due to its simple yet flexible parametric form, which requires the estimation of only a few parameters, ie, the scatter matrix and the shape parameter Existing fixed-point (FP) algorithms provide an easy to implement method for estimating the scatter matrix, but are known to fail, giving highly inaccurate results, when the value of the shape parameter increases Since many applications require flexible estimation of the shape parameter, we propose a new FP algorithm, Riemannian averaged FP (RA-FP), which can effectively estimate the scatter matrix for any value of the shape parameter We provide the mathematical justification of the convergence of the RA-FP algorithm based on the Riemannian geometry of the space of symmetric positive definite matrices We also show using numerical simulations that the RA-FP algorithm is invariant to the initialization of the scatter matrix and provides significantly improved performance over existing FP and method-of-moments (MoM) algorithms for the estimation of the scatter matrix

Independent vector analysis for SSVEP signal enhancement

Abstract: Steady state visual evoked potentials (SSVEP) have been identified as a highly viable solution for brain computer interface (BCI) systems. The SSVEP is observed in the scalp-based recordings of electroencephalogram (EEG) signals, and is one component buried amongst the normal brain signals and complex noise. By taking advantage of sample diversity, higher order statistics and statistical dependencies associated with the analysis of multiple datasets, independent vector analysis (IVA) can be used to enhance the detection of the SSVEP signal content. In this paper, we present a novel method for detecting SSVEP signals by treating each EEG signal as a stand alone data set. IVA is used to exploit the correlation across the estimated sources, as well as statistical diversity within datasets to enhance SSVEP detection, offering a significant improvement over averaging based methods for the detection of the SSVEP signal.

Density estimation by entropy maximization with kernels

Abstract: The estimation of a probability density function is one of the most fundamental problems in statistics. The goal is achieving a desirable balance between flexibility while maintaining as simple a form as possible to allow for generalization, and efficient implementation. In this paper, we use the maximum entropy principle to achieve this goal and present a density estimator that is based on two types of approximation. We employ both global and local measuring functions, where Gaussian kernels are used as local measuring functions. The number of the Gaussian kernels is estimated by the minimum description length criterion, and the parameters are estimated by expectation maximization and a new probability difference measure. Experimental results show the flexibility and desirable performance of this new method.

A data-driven solution for abandoned object detection: Advantages of multiple types of diversity

Abstract: The automated detection of abandoned objects is a quickly developing and widely researched field in video processing with specific application to automated surveillance. In the recent years, a number of approaches have been proposed to automatically detect abandoned objects. However, these techniques require prior knowledge of certain properties of the object such as its shape and color, to classify the foreground objects as abandoned object. The performance of tracking-based approaches degrades in complex scenes, i.e., when the abandoned object is occluded or in the case of crowding. In this paper, we propose a data-driven approach based on independent component analysis (ICA) for object detection. We demonstrate the success of the proposed ICA-based methodology with examples of videos with complex scenarios. We also show that algorithm choice plays an important role in performance, in particular when multiple types of diversities are taken into account and demonstrate the importance of order selection.

Independent vector analysis by entropy rate bound minimization

Abstract: An extension of independent component analysis from one to multiple datasets, independent vector analysis, has recently become a subject of significant research interest. Since in many applications, latent sources are non-Gaussian, have sample dependence, and have dependence across multiple data sets, it is desirable to exploit all these properties jointly. Mutual information rate, which leads to the minimization of entropy rate, provides a natural cost for the task. In this paper, we present a new algorithm by using an effective entropy rate estimator, which takes all these properties into account. Experimental results show that the new method accounts for these properties effectively.

Multimodal Data Fusion [Scanning the Issue]

Abstract: The articles in this special issue focus on multimodal data fusion. These papers provide a review of current approaches and results on multimodal data fusion from different disciplines, and by including the required background in each domain, aims to provide a common forum for the exchange of ideas in this very active field of research.

Multivariate Fusion of EEG and Functional MRI Data Using ICA: Algorithm Choice and Performance Analysis

Abstract: It has become common for neurological studies to gather data from multiple modalities, since the modalities examine complementary aspects of neural activity. Functional magnetic resonance imaging fMRI and electroencephalogram EEG data, in particular, enable the study of functional changes within the brain at different temporal and spatial scales; hence their fusion has received much attention. Joint independent component analysis jICA enables symmetric and fully multivariate fusion of these modalities and is thus one of the most widely used methods. In its application to jICA, Infomax has been the widely used, however the relative performance of Infomax is rarely shown on real neurological data, since the ground truth is not known. We propose the use of number of voxels in physically meaningful masks and statistical significance to assess algorithm performance of ICA for jICA on real data and show that entropy bound minimization EBM provides a more attractive solution for jICA of EEG and fMRI.