There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Independent component models have gained increasing interest in various fields of applications in recent years. The basic independent component model is a semiparametric model assuming that a p-variate observed random vector is a linear transformation of an unobserved vector of p independent latent variables. This linear transformation is given by an unknown mixing matrix, and one of the main objectives of independent component analysis (ICA) is to estimate an unmixing matrix by means of which the latent variables can be recovered. In this article, we discuss the basic independent component model in detail, define the concepts and analysis tools carefully, and consider two families of ICA estimates. The statistical properties (consistency, asymptotic normality, efficiency, robustness) of the estimates can be analyzed and compared via the so called gain matrices. Some extensions of the basic independent component model, such as models with additive noise or models with dependent observations, are briefly discussed. The article ends with a short example.


Keywords:

blind source separation;
fastICA;
independent component model;
independent subspace analysis;
mixing matrix;
overcomplete ICA;
undercomplete ICA;
unmixing matrix

Independent Component Analysis

Many problems in the sciences and engineering can be rephrased as optimization problems on matrix search spaces endowed with a so-called manifold structure. This book shows how to exploit the special structure of such problems to develop efficient numerical algorithms. It places careful emphasis on both the numerical formulation of the algorithm and its differential geometric abstraction--illustrating how good algorithms draw equally from the insights of differential geometry, optimization, and numerical analysis. Two more theoretical chapters provide readers with the background in differential geometry necessary to algorithmic development. In the other chapters, several well-known optimization methods such as steepest descent and conjugate gradients are generalized to abstract manifolds. The book provides a generic development of each of these methods, building upon the material of the geometric chapters. It then guides readers through the calculations that turn these geometrically formulated methods into concrete numerical algorithms. The state-of-the-art algorithms given as examples are competitive with the best existing algorithms for a selection of eigenspace problems in numerical linear algebra. Optimization Algorithms on Matrix Manifolds offers techniques with broad applications in linear algebra, signal processing, data mining, computer vision, and statistical analysis. It can serve as a graduate-level textbook and will be of interest to applied mathematicians, engineers, and computer scientists.

http://www.eeci-institute.eu/GSC2011/Photos-EECI/EECI-GSC-2011-M5/book_AMS.pdf

Optimization Algorithms on Matrix Manifolds

Independent Component Analysis.

Separation of sources consists of recovering a set of signals of which only instantaneous linear mixtures are observed. In many situations, no a priori information on the mixing matrix is available: The linear mixture should be "blindly" processed. This typically occurs in narrowband array processing applications when the array manifold is unknown or distorted. This paper introduces a new source separation technique exploiting the time coherence of the source signals. In contrast with other previously reported techniques, the proposed approach relies only on stationary second-order statistics that are based on a joint diagonalization of a set of covariance matrices. Asymptotic performance analysis of this method is carried out; some numerical simulations are provided to illustrate the effectiveness of the proposed method.

A blind source separation technique using second-order statistics

Blind system identification (BSI) is a fundamental signal processing technology aimed at retrieving a system's unknown information from its output only. This technology has a wide range of possible applications such as mobile communications, speech reverberation cancellation, and blind image restoration. This paper reviews a number of recently developed concepts and techniques for BSI, which include the concept of blind system identifiability in a deterministic framework, the blind techniques of maximum likelihood and subspace for estimating the system's impulse response, and other techniques for direct estimation of the system input.

/pdf/blind-system-identification-4eaj4ocrup.pdf

Blind system identification

We construct a new family of linear space-time (ST) block codes by the combination of rotated constellations and the Hadamard transform, and we prove them to achieve the full transmit diversity over a quasi-static or fast fading channels. The proposed codes transmit at a normalized rate of 1 symbol/s. When the number of transmit antennas n=1, 2, or n is a multiple of four, we spread a rotated version of the information symbol vector by the Hadamard transform and send it over n transmit antennas and n time periods; for other values of n, we construct the codes by sending the components of a rotated version of the information symbol vector over the diagonal of an n /spl times/ n ST code matrix. The codes maintain their rate, diversity, and coding gains for all real and complex constellations carved from the complex integers ring Z [i], and they outperform the codes from orthogonal design when using complex constellations for n > 2. The maximum-likelihood (ML) decoding of the proposed codes can be implemented by the sphere decoder at a moderate complexity. It is shown that using the proposed codes in a multiantenna system yields good performances with high spectral efficiency and moderate decoding complexity.

Diagonal algebraic space-time block codes

The problem of blind identification of p-inputs/q-outputs FIR transfer functions is addressed. Existing subspace identification methods derived for p=1 are first reformulated. In particular, the links between the noise subspace of a certain covariance matrix of the output signals (on which subspace methods build on) and certain rational subspaces associated with the transfer function to be identified are elucidated. Based on these relations, we study the behavior of the subspace method in the case where the order of the transfer function is overestimated. Next, an asymptotic performance analysis of this estimation method is carried out. Consistency and asymptotical normality of the estimates is established. A closed-form expression for the asymptotic covariance of the estimates is given. Numerical simulations and investigations are presented to demonstrate the potential of the subspace method. Finally, we take advantage of our new reformulation to discuss the extension of the subspace method to the case p>1. We show where the difficulties lie, and we briefly indicate how to solve the corresponding problems. The possible connections with classical approaches for MA model estimations are also outlined.

A subspace algorithm for certain blind identification problems

Blind channel identification methods based on the oversampled channel output are a problem of current theoretical and practical interest. In this paper, we introduce a second-order blind identification technique based on a linear prediction approach. In contrast to eigenstructure-based methods, it will be shown that the linear prediction error method is "robust" to order overdetermination. An asymptotic performance analysis of the proposed estimation method is carried out, consistency and asymptotic normality of the estimates is established. A closed-form expression for the asymptotic covariance of the estimates is given. Numerical simulations and investigations are finally presented to demonstrate the potential and the "robustness" of the proposed method.

Karim Abed-Meraim

Papers

A blind source separation technique using second-order statistics

Blind system identification

Diagonal algebraic space-time block codes

A subspace algorithm for certain blind identification problems

Prediction error method for second-order blind identification