There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Independent component models have gained increasing interest in various fields of applications in recent years. The basic independent component model is a semiparametric model assuming that a p-variate observed random vector is a linear transformation of an unobserved vector of p independent latent variables. This linear transformation is given by an unknown mixing matrix, and one of the main objectives of independent component analysis (ICA) is to estimate an unmixing matrix by means of which the latent variables can be recovered. In this article, we discuss the basic independent component model in detail, define the concepts and analysis tools carefully, and consider two families of ICA estimates. The statistical properties (consistency, asymptotic normality, efficiency, robustness) of the estimates can be analyzed and compared via the so called gain matrices. Some extensions of the basic independent component model, such as models with additive noise or models with dependent observations, are briefly discussed. The article ends with a short example.


Keywords:

blind source separation;
fastICA;
independent component model;
independent subspace analysis;
mixing matrix;
overcomplete ICA;
undercomplete ICA;
unmixing matrix

Independent Component Analysis

Many problems in the sciences and engineering can be rephrased as optimization problems on matrix search spaces endowed with a so-called manifold structure. This book shows how to exploit the special structure of such problems to develop efficient numerical algorithms. It places careful emphasis on both the numerical formulation of the algorithm and its differential geometric abstraction--illustrating how good algorithms draw equally from the insights of differential geometry, optimization, and numerical analysis. Two more theoretical chapters provide readers with the background in differential geometry necessary to algorithmic development. In the other chapters, several well-known optimization methods such as steepest descent and conjugate gradients are generalized to abstract manifolds. The book provides a generic development of each of these methods, building upon the material of the geometric chapters. It then guides readers through the calculations that turn these geometrically formulated methods into concrete numerical algorithms. The state-of-the-art algorithms given as examples are competitive with the best existing algorithms for a selection of eigenspace problems in numerical linear algebra. Optimization Algorithms on Matrix Manifolds offers techniques with broad applications in linear algebra, signal processing, data mining, computer vision, and statistical analysis. It can serve as a graduate-level textbook and will be of interest to applied mathematicians, engineers, and computer scientists.

http://www.eeci-institute.eu/GSC2011/Photos-EECI/EECI-GSC-2011-M5/book_AMS.pdf

Optimization Algorithms on Matrix Manifolds

Independent Component Analysis.

In this paper, we consider the problem of blind identification of FIR systems using the cross-relations (CR) method first introduced in H. Liu et al. (1994). Our contribution in this paper is as follows: (i) we introduce an extended formulation of the CR identification criterion, which generalizes the standard CR criterion used in H. Liu et al. (1994). It can be shown that many existing multichannel blind identification methods belong to the class of generalized CR methods; (ii) we introduce a new identification method referred to as minimum cross-relations (MCR) method, which exploits with minimum redundancy the spatial diversity among the channel outputs. Simulation-based performance analysis of the MCR method and comparisons with CR method are also presented; (iii) Then, we present a modified version of the MCR referred to as the "unbiased MCR" (UMCR) method that leads to unbiased estimation of the channel parameters and better estimation performances without need of noise whitening as in the MCR. (iv) Finally, we discuss the multiinput case and show how additional difficulties arise due to the nonlinear parameterization of the noise vectors in terms of the channel parameters.

/pdf/blind-system-identification-using-cross-relation-methods-26lkqzk2h6.pdf

Blind system identification using cross-relation methods: further results and developments

In this article, we present a two-stage blind source separation (BSS) algorithm for robot audition. The first stage consists in a fixed beamforming preprocessing to reduce the reverberation and the environmental noise. Since we are in a robot audition context, the manifold of the sensor array in this case is hard to model due to the presence of the head of the robot, so we use pre-measured head related transfer functions (HRTFs) to estimate the beamforming filters. The use of the HRTF to estimate the beamformers allows to capture the effect of the head on the manifold of the microphone array. The second stage is a BSS algorithm based on a sparsity criterion which is the minimization of the l1 norm of the sources. We present different configuration of our algorithm and we show that it has promising results and that the fixed beamforming preprocessing improves the separation results.

/pdf/blind-source-separation-for-robot-audition-using-fixed-hrtf-2fn7rwfx4c.pdf

Blind source separation for robot audition using fixed HRTF beamforming

This letter proposes a novel technique for the blind separation of autoregressive (AR) sources. The latter relies on the joint diagonalization (JD) of appropriate AR matrix coefficients of the observed signals and can be applied to the separation of statistically dependent sources. The developed algorithm is referred to as ‘DARSS-JD’ (for Dependent AR Source Separation using JD). Through the simulation experiments, DARSS-JD is shown to overcome existing second order separation methods with a relatively moderate computational cost.

Separation of Dependent Autoregressive Sources Using Joint Matrix Diagonalization

In this work, we present the Damped and Delayed Sinusoidal (DDS) model: a generalization of the sinusoidal model. This model takes into account an angular frequency, a damping factor, a phase, an amplitude, and a time-delay parameter for each component. Two algorithms are introduced for the DDS parameter estimation using a subband processing approach. Finally, we derive the Crame/spl acute/r-Rao Bound (CRB) expression for the DDS model and a simulation-based performance analysis in the context of a noisy fast time-varying synthetic signal and in the audio transient signal modeling context.

/pdf/damped-and-delayed-sinusoidal-model-for-transient-signals-2w55hlrkiz.pdf

Damped and delayed sinusoidal model for transient signals

La s eparation aveugle de sources consiste a retrouver un ensemble de signaux statistiquement ind ependants a par-tir de m elanges observ es. Cet article introduit une nou-velle technique de s eparation de sources exploitant la coh erence temporelle des signaux sources. A la dii erence des techniques classiques de s eparation, cette approche utilisant seulement les statistiques du second-ordre est bas ee sur \la diagonalisation conjointe" des matrices de corr elation temporelle. Des simulations num eriques illus-trent la validit e de la m ethode dans des contextes diiciles. Abstract Blind separation of sources consists in recovering a set of statistically independent signals whose only instantaneous mixtures are observed. Such mixtures occur in narrow band array data which can then be processed without knowing the array manifold (blindness). This paper introduces a new source separation technique exploiting the possible time coherence of the source signals. In contrast to other previously reported techniques, the proposed approach relies only on second-order statistics, being based on a `joint diagonalization' of correlation matrices. The eeectiveness of the method in diicult contexts is illustrated by numerical simulations.

Karim Abed-Meraim

Papers

Blind system identification using cross-relation methods: further results and developments

Blind source separation for robot audition using fixed HRTF beamforming

Separation of Dependent Autoregressive Sources Using Joint Matrix Diagonalization

Damped and delayed sinusoidal model for transient signals

Séparation aveugle au second ordre de sources corrélées