Multiple-input multiple-output (MIMO) channels provide an abstract and unified representation of different physical communication systems, ranging from multi-antenna wireless channels to wireless digital subscriber line systems. They have the key property that several data streams can be simultaneously established.

In general, the design of communication systems for MIMO channels is quite involved (if one can assume the use of sufficiently long and good codes, then the problem formulation simplifies drastically). The first difficulty lies on how to measure the global performance of such systems given the tradeoff on the performance among the different data streams. Once the problem formulation is defined, the resulting mathematical problem is typically too complicated to be optimally solved as it is a matrix-valued nonconvex optimization problem. This design problem has been studied for the past three decades (the first papers dating back to the 1970s) motivated initially by cable systems and more recently by wireless multi-antenna systems. The approach was to choose a specific global measure of performance and then to design the system accordingly, either optimally or suboptimally, depending on the difficulty of the problem.

This text presents an up-to-date unified mathematical framework for the design of point-to-point MIMO transceivers with channel state information at both sides of the link according to an arbitrary cost function as a measure of the system performance. In addition, the framework embraces the design of systems with given individual performance on the data streams.

Majorization theory is the underlying mathematical theory on which the framework hinges. It allows the transformation of the originally complicated matrix-valued nonconvex problem into a simple scalar problem. In particular, the additive majorization relation plays a key role in the design of linear MIMO transceivers (i.e., a linear precoder at the transmitter and a linear equalizer at the receiver), whereas the multiplicative majorization relation is the basis for nonlinear decision-feedback MIMO transceivers (i.e., a linear precoder at the transmitter and a decision-feedback equalizer at the receiver).

https://www.nowpublishers.com/article/DownloadSummary/CIT-018

Mimo Transceiver Design Via Majorization Theory

We present parallel and sequential dense QR factorization algorithms that are both optimal (up to polylogarithmic factors) in the amount of communication they perform and just as stable as Householder QR. We prove optimality by deriving new lower bounds for the number of multiplications done by “non-Strassen-like” QR, and using these in known communication lower bounds that are proportional to the number of multiplications. We not only show that our QR algorithms attain these lower bounds (up to polylogarithmic factors), but that existing LAPACK and ScaLAPACK algorithms perform asymptotically more communication. We derive analogous communication lower bounds for LU factorization and point out recent LU algorithms in the literature that attain at least some of these lower bounds. The sequential and parallel QR algorithms for tall and skinny matrices lead to significant speedups in practice over some of the existing algorithms, including LAPACK and ScaLAPACK, for example, up to 6.7 times over ScaLAPACK. A performance model for the parallel algorithm for general rectangular matrices predicts significant speedups over ScaLAPACK.

/pdf/communication-optimal-parallel-and-sequential-qr-and-lu-1ldw6a0wk4.pdf

Communication-optimal Parallel and Sequential QR and LU Factorizations

We present parallel and sequential dense QR factorization algorithms that are both optimal (up to polylogarithmic factors) in the amount of communication they perform, and just as stable as Householder QR. 
We prove optimality by extending known lower bounds on communication bandwidth for sequential and parallel matrix multiplication to provide latency lower bounds, and show these bounds apply to the LU and QR decompositions. We not only show that our QR algorithms attain these lower bounds (up to polylogarithmic factors), but that existing LAPACK and ScaLAPACK algorithms perform asymptotically more communication. We also point out recent LU algorithms in the literature that attain at least some of these lower bounds.

Communication-optimal parallel and sequential QR and LU factorizations

http://www.sal.ufl.edu/yjiang/papers/gmd.pdf

The geometric mean decomposition

The traditional metric for the efficiency of a numerical algorithm has been the number of arithmetic operations it performs. Technological trends have long been reducing the time to perform an arithmetic operation, so it is no longer the bottleneck in many algorithms; rather, communication, or moving data, is the bottleneck. This motivates us to seek algorithms that move as little data as possible, either between levels of a memory hierarchy or between parallel processors over a network. In this paper we summarize recent progress in three aspects of this problem. First we describe lower bounds on communication. Some of these generalize known lower bounds for dense classical (O(n3)) matrix multiplication to all direct methods of linear algebra, to sequential and parallel algorithms, and to dense and sparse matrices. We also present lower bounds for Strassen-like algorithms, and for iterative methods, in particular Krylov subspace methods applied to sparse matrices. Second, we compare these lower bounds to widely used versions of these algorithms, and note that these widely used algorithms usually communicate asymptotically more than is necessary. Third, we identify or invent new algorithms for most linear algebra problems that do attain these lower bounds, and demonstrate large speed-ups in theory and practice.

Communication lower bounds and optimal algorithms for numerical linear algebra

Set-theoretic solutions of the Yang–Baxter equation and new classes of R-matrices

An error analysis result is given for classical Gram–Schmidt factorization of a full rank matrix A into A =  QR where Q is left orthogonal (has orthonormal columns) and R is upper triangular. The work presented here shows that the computed R satisfies RT R = AT A +  E where E is an appropriately small backward error, but only if the diagonals of R are computed in a manner similar to Cholesky factorization of the normal equations matrix. At the end of the article, implications for classical Gram–Schmidt with reorthogonalization are noted.A similar result is stated in Giraud et al. (Numer Math 101(1):87–100, 2005). However, for that result to hold, the diagonals of R must be computed in the manner recommended in this work.

A note on the error analysis of classical Gram–Schmidt

In this paper we study the numerical properties of several orthogonalization schemes where the inner product is induced by a nontrivial symmetric and positive definite matrix. We analyze the effect of its conditioning on the factorization and the loss of orthogonality between vectors computed in finite precision arithmetic. We consider the implementation based on the backward stable eigendecomposition, modified and classical Gram–Schmidt algorithms, Gram–Schmidt process with reorthogonalization as well as the implementation motivated by the AINV approximate inverse preconditioner.

Numerical stability of orthogonalization methods with a non-standard inner product

A new reorthogonalized block classical Gram--Schmidt algorithm is proposed that factorizes a full column rank matrix $A$ into $A=QR$ where $Q$ is left orthogonal (has orthonormal columns) and $R$ is upper triangular and nonsingular. 
With appropriate assumptions on the diagonal blocks of $R$, the algorithm, when implemented in floating point arithmetic with machine unit $\macheps$, produces $Q$ and $R$ such that $\| I- Q^{T} Q \|_2 =O(\macheps)$ and $\| A-QR \|_2 =O(\macheps \| A \|_2)$. The resulting bounds also improve a previous bound by Giraud et al. [Num. Math., 101(1):87-100,\ 2005] on the CGS2 algorithm originally developed by Abdelmalek [BIT, 11(4):354--367,\ 1971]. 
\medskip Keywords: Block matrices, Q--R factorization, Gram-Schmidt process, Condition numbers, Rounding error analysis.

Reorthogonalized Block Classical Gram--Schmidt

We study numerical properties of Clenshaw's algorithm for summing the series w = ∑n = 0Nbnpn where pn satisfy the linear three-term recurrence relation. We prove that under natural assumptions Clenshaw's algorithm is backward stable with respect to the data bn, n = 0,N.

Alicja Smoktunowicz

Papers

Set-theoretic solutions of the Yang–Baxter equation and new classes of R-matrices

A note on the error analysis of classical Gram–Schmidt

Numerical stability of orthogonalization methods with a non-standard inner product

Reorthogonalized Block Classical Gram--Schmidt

Backward Stability of Clenshaw's Algorithm