Information Theory and Reliable Communication

Most of the signal processing that we will study in this course involves local operations on a signal, namely transforming the signal by applying linear combinations of values in the neighborhood of each sample point. You are familiar with such operations from Calculus, namely, taking derivatives and you are also familiar with this from optics namely blurring a signal. We will be looking at sampled signals only. Let's start with a few basic examples. Local difference Suppose we have a 1D image and we take the local difference of intensities, DI(x) = 1 2 (I(x + 1) − I(x − 1)) which give a discrete approximation to a partial derivative. (We compute this for each x in the image.) What is the effect of such a transformation? One key idea is that such a derivative would be useful for marking positions where the intensity changes. Such a change is called an edge. It is important to detect edges in images because they often mark locations at which object properties change. These can include changes in illumination along a surface due to a shadow boundary, or a material (pigment) change, or a change in depth as when one object ends and another begins. The computational problem of finding intensity edges in images is called edge detection. We could look for positions at which DI(x) has a large negative or positive value. Large positive values indicate an edge that goes from low to high intensity, and large negative values indicate an edge that goes from high to low intensity. Example Suppose the image consists of a single (slightly sloped) edge:

http://ieeexplore.ieee.org/iel5/5/31307/01456462.pdf

Linear systems

Spectrum sensing is a key component of the cognitive radio paradigm. Primary signals are typically detected with uncalibrated receivers at signal-to-noise ratios (SNRs) well below decodability levels. Multiantenna detectors exploit spatial independence of receiver thermal noise to boost detection performance and robustness. We study the problem of detecting a Gaussian signal with rank-P unknown spatial covariance matrix in spatially uncorrelated Gaussian noise with unknown covariance using multiple antennas. The generalized likelihood ratio test (GLRT) is derived for two scenarios. In the first one, the noises at all antennas are assumed to have the same (unknown) variance, whereas in the second, a generic diagonal noise covariance matrix is allowed in order to accommodate calibration uncertainties in the different antenna frontends. In the latter case, the GLRT statistic must be obtained numerically, for which an efficient method is presented. Furthermore, for asymptotically low SNR, it is shown that the GLRT does admit a closed form, and the resulting detector performs well in practice. Extensions are presented in order to account for unknown temporal correlation in both signal and noise, as well as frequency-selective channels.

Detection of Rank- $P$ Signals in Cognitive Radio Networks With Uncalibrated Multiple Antennas

This work addresses the problem of deciding whether a set of realizations of a vector-valued time series with unknown temporal correlation are spatially correlated or not. For wide sense stationary (WSS) Gaussian processes, this is a problem of deciding between two different power spectral density matrices, one of them diagonal. Specifically, we show that for arbitrary Gaussian processes (not necessarily WSS) the generalized likelihood ratio test (GLRT) is given by the quotient between the determinant of the sample space-time covariance matrix and the determinant of its block-diagonal version. Furthermore, for WSS processes, we present an asymptotic frequency-domain approximation of the GLRT which is given by a function of the Hadamard ratio (quotient between the determinant of a matrix and the product of the elements of the main diagonal) of the estimated power spectral density matrix. The Hadamard ratio is known to be the GLRT detector for vector-valued random variables and, therefore, what this paper shows is how frequency-dependent Hadamard ratios must be merged into a single test statistic when the vector-valued random variable is replaced by a vector-valued time series with temporal correlation. For bivariate time series, the derived frequency domain detector can be rewritten as a function of the well-known magnitude squared coherence (MSC) spectrum, which suggests a straightforward extension of the MSC spectrum to the general case of multivariate time series. Finally, the performance of the proposed method is illustrated by means of simulations.

Detection of Spatially Correlated Gaussian Time Series

For the engineering community, Gray's tutorial monograph on Toeplitz and circulant matrices has been, and remains, the best elementary introduction to the Szego theory on large Toeplitz matrices. In this paper, the most important results of the cited monograph are generalized to block Toeplitz (BT) matrices by maintaining the same mathematical tools used by Gray, that is, by using asymptotically equivalent sequences of matrices. As applications of these results, the geometric minimum mean square error (MMSE) for both an infinite-length multivariate linear predictor and an infinite-length decision feedback equalizer (DFE) for multiple-input-multiple-output (MIMO) channels, are obtained as a limit of the corresponding finite-length cases. Similarly, a short derivation of the well-known capacity of a time-invariant MIMO Gaussian channel with intersymbol interference (ISI) and fixed input covariance matrix is also presented.

Asymptotically Equivalent Sequences of Matrices and Hermitian Block Toeplitz Matrices With Continuous Symbols: Applications to MIMO Systems

The present monograph studies the asymptotic behaviour of eigenvalues, products and functions of block Toeplitz matrices generated by the Fourier coefficients of a continuous matrix-valued function. This study is based on the concept of asymptotically equivalent sequences of non-square matrices. The asymptotic results on block Toeplitz matrices obtained are applied to vector asymptotically wide sense stationary processes. Therefore, this monograph is a generalization to block Toeplitz matrices of the Gray monograph entitled “Toeplitz and circulant matrices: A review”, which was published in the second volume of Foundations and Trends in Communications and Information Theory, and which is the simplest and most famous introduction to the asymptotic theory on Toeplitz matrices.

/pdf/block-toeplitz-matrices-asymptotic-results-and-applications-1f5686oncf.pdf

Block Toeplitz Matrices: Asymptotic Results and Applications

Functions of banded Hermitian block Toeplitz matrices in signal processing

We consider a multiple access relay channel where two sources S1 and S2 send their data to a common destination node aided by a shared relay node. In this paper we extend previous work on this topic by assuming that the output of both information sources are correlated with memory. Furthermore, we consider frequency-selective fading channels, which introduce inter-symbol interference among the transmitted symbols. In this context, we propose an iterative receiver which comprises turbo maximum a posteriori (MAP) equalization and distributed source-channel- network decoding in an iterative fashion. Various relaying strategies such as network coding or time multiplexing are analyzed in detail. In addition, a suboptimal but less complex minimum mean square error (MMSE) linear equalizer is also utilized to account for communication scenarios with constrained complexity. Simulation results show that network coding based strategy outperforms, in all cases, the other considered forwarding schemes.

Jesús Gutiérrez-Gutiérrez

Papers

Asymptotically Equivalent Sequences of Matrices and Hermitian Block Toeplitz Matrices With Continuous Symbols: Applications to MIMO Systems

Block Toeplitz Matrices: Asymptotic Results and Applications

Functions of banded Hermitian block Toeplitz matrices in signal processing

On Combining Distributed Joint Source-Channel-Network Coding and Turbo Equalization in Multiple Access Relay Networks

Mass concentration in quasicommutators of Toeplitz matrices