The Theory of Matrices. By F R. Gantmacher. Two volumes, pp. 374 and 276. 1959. (Translated from the Russian by K. A. Hirsch; Chelsea Publishing Company, New York)

Large linear systems of saddle point type arise in a wide variety of applications throughout computational science and engineering. Due to their indefiniteness and often poor spectral properties, such linear systems represent a significant challenge for solver developers. In recent years there has been a surge of interest in saddle point problems, and numerous solution techniques have been proposed for this type of system. The aim of this paper is to present and discuss a large selection of solution methods for linear systems in saddle point form, with an emphasis on iterative methods for large and sparse problems.

/pdf/numerical-solution-of-saddle-point-problems-l583iob5c2.pdf

Numerical solution of saddle point problems

Most of the signal processing that we will study in this course involves local operations on a signal, namely transforming the signal by applying linear combinations of values in the neighborhood of each sample point. You are familiar with such operations from Calculus, namely, taking derivatives and you are also familiar with this from optics namely blurring a signal. We will be looking at sampled signals only. Let's start with a few basic examples. Local difference Suppose we have a 1D image and we take the local difference of intensities, DI(x) = 1 2 (I(x + 1) − I(x − 1)) which give a discrete approximation to a partial derivative. (We compute this for each x in the image.) What is the effect of such a transformation? One key idea is that such a derivative would be useful for marking positions where the intensity changes. Such a change is called an edge. It is important to detect edges in images because they often mark locations at which object properties change. These can include changes in illumination along a surface due to a shadow boundary, or a material (pigment) change, or a change in depth as when one object ends and another begins. The computational problem of finding intensity edges in images is called edge detection. We could look for positions at which DI(x) has a large negative or positive value. Large positive values indicate an edge that goes from low to high intensity, and large negative values indicate an edge that goes from high to low intensity. Example Suppose the image consists of a single (slightly sloped) edge:

http://ieeexplore.ieee.org/iel5/5/31307/01456462.pdf

Linear systems

"Matrix Mathematics" is a reference work for users of matrices in all branches of engineering, science, and applied mathematics. This book brings together a vast body of results on matrix theory for easy reference and immediate application. Each chapter begins with the development of relevant background theory followed by a large collection of specialized results. Hundreds of identities, inequalities, and matrix facts are stated rigorously and clearly with cross references, citations to the literature, and illuminating remarks. Twelve chapters cover all of the major topics in matrix theory: preliminaries; basic matrix properties; matrix classes and transformations; matrix polynomials and rational transfer functions; matrix decompositions; generalized inverses; Kronecker and Schur algebra; positive-semidefinite matrices; norms; functions of matrices and their derivatives; the matrix exponential and stability theory; and linear systems and control theory. A detailed list of symbols, a summary of notation and conventions, an extensive bibliography with author index, and an extensive index are provided for ease of use. The book will be useful for students at both the undergraduate and graduate levels, as well as for researchers and practitioners in all branches of engineering, science, and applied mathematics.

/pdf/matrix-mathematics-theory-facts-and-formulas-with-8swhnd659u.pdf

Matrix Mathematics: Theory, Facts, and Formulas with Application to Linear Systems Theory

Determine the value of the variable in each equation.

Basic Algebra = 代数学入門

Let $A$ belong to an automorphism group, Lie algebra, or Jordan algebra of a scalar product. When $A$ is factored, to what extent do the factors inherit structure from $A$? We answer this question for the principal matrix square root, the matrix sign decomposition, and the polar decomposition. For general $A$, we give a simple derivation and characterization of a particular generalized polar decomposition, and we relate it to other such decompositions in the literature. Finally, we study eigendecompositions and structured singular value decompositions, considering in particular the structure in eigenvalues, eigenvectors, and singular values that persists across a wide range of scalar products.
A key feature of our analysis is the identification of two particular classes of scalar products, termed unitary and orthosymmetric, which serve to unify assumptions for the existence of structured factorizations. A variety of different characterizations of these scalar product classes are given.

/pdf/structured-factorizations-in-scalar-product-spaces-30d9ojjtk9.pdf

Structured Factorizations in Scalar Product Spaces

For which functions $f$ does $A\in\mathbb{G} \Rightarrow f(A)\in\mathbb{G}$ when $\mathbb{G}$ is the matrix automorphism group associated with a bilinear or sesquilinear form? For example, if $A$ is symplectic when is $f(A)$ symplectic? We show that group structure is preserved precisely when $f(A^{-1}) = f(A)^{-1}$ for bilinear forms and when $f(A^{-*}) = f(A)^{-*}$ for sesquilinear forms. Meromorphic functions that satisfy each of these conditions are characterized. Related to structure preservation is the condition $f(\overline{A}) = \overline{f(A)}$, and analytic functions and rational functions satisfying this condition are also characterized. These results enable us to characterize all meromorphic functions that map every $\mathbb{G}$ into itself as the ratio of a polynomial and its ``reversal,'' up to a monomial factor and conjugation.
The principal square root is an important example of a function that preserves every automorphism group $\mathbb{G}$. By exploiting the matrix sign function, a new family of coupled iterations for the matrix square root is derived. Some of these iterations preserve every $\mathbb{G}$; all of them are shown, via a novel Frechet derivative-based analysis, to be numerically stable.
A rewritten form of Newton's method for the square root of $A\in\mathbb{G}$ is also derived. Unlike the original method, this new form has good numerical stability properties, and we argue that it is the iterative method of choice for computing $A^{1/2}$ when $A\in\mathbb{G}$. Our tools include a formula for the sign of a certain block $2\times 2$ matrix, the generalized polar decomposition along with a wide class of iterations for computing it, and a connection between the generalized polar decomposition of $I+A$ and the square root of $A\in\mathbb{G}$.

/pdf/functions-preserving-matrix-groups-and-iterations-for-the-nzy9oj1sx1.pdf

Functions Preserving Matrix Groups and Iterations for the Matrix Square Root

An extensive and unified collection of structure-preserving transformations is pre- sented and organized for easy reference. The structures involved arise in the context of a non- degenerate bilinear or sesquilinear form on R n or C n . A variety of transformations belonging to the automorphism groups of these forms, that imitate the action of Givens rotations, Householder reflec- tors, and Gauss transformations are constructed. Transformations for performing structured scaling actions are also described. The matrix groups considered in this paper are the complex orthogonal, real, complex and conjugate symplectic, real perplectic, real and complex pseudo-orthogonal, and pseudo-unitary groups. In addition to deriving new transformations, this paper collects and unifies existing structure-preserving tools.

/pdf/structured-tools-for-structured-matrices-59gawgvipp.pdf

Structured tools for structured matrices

http://unix.cc.wmich.edu/~mackey/mmmm7.pdf

D. Steven Mackey

Papers

Structured Factorizations in Scalar Product Spaces

Functions Preserving Matrix Groups and Iterations for the Matrix Square Root

Structured tools for structured matrices

Möbius transformations of matrix polynomials

Hamilton and Jacobi come full circle: Jacobi algorithms for structured Hamiltonian eigenproblems