In this survey paper, we describe how strands of work that are important in two different fields, matrix theory and complex function theory, have come together in some work on fast computational algorithms for matrices with what we call displacement structure. In particular, a fast triangularization procedure can be developed for such matrices, generalizing in a striking way an algorithm presented by Schur (1917) [J. Reine Angew. Math., 147 (1917), pp. 205–232] in a paper on checking when a power series is bounded in the unit disc. This factorization algorithm has a surprisingly wide range of significant applications going far beyond numerical linear algebra. We mention, among others, inverse scattering, analytic and unconstrained rational interpolation theory, digital filter design, adaptive filtering, and state-space least-squares estimation.

Displacement structure: theory and applications

Nova Science Publishers

Fast O(n 2 ) implementation of Gaussian elimination with partial pivoting is designed for matrices possessing Cauchy-like displacement structure. We show how Toeplitz-like, Toeplitz-plus-Hankel-like and Vandermonde-like matrices can be transformed into Cauchy-like matrices by using Discrete Fourier, Cosine or Sine Transform matrices. In particular this allows us to propose a new fast O(n 2 ) Toeplitz solver GKO, which incorporates partial pivoting. A large set of numerical examples showed that GKO demonstrated stable numerical behavior and can be recommended for solving linear systems, especially with nonsymmetric, indefinite and ill-conditioned positive definite Toeplitz matrices. It is also useful for block Toeplitz and mosaic Toeplitz ( Toeplitz-block ) matrices. The algorithms proposed in this paper suggest an attractive alternative to look-ahead approaches, where one has to jump over ill-conditioned leading submatrices, which in the worst case requires O(n 3 ) operations.

https://www.ams.org/mcom/1995-64-212/S0025-5718-1995-1312096-X/S0025-5718-1995-1312096-X.pdf

Fast Gaussian elimination with partial pivoting for matrices with displacement structure

Superlinear preconditioners (those that provide a proper cluster at 1) are very important for the cg-like methods since they make these methods converge superlinearly. As is well known, for Toeplitz matrices generated by a continuous symbol, many circulant and circulant-like (related to different matrix algebras) preconditioners were proved to be superlinear. In contrast, for multilevel Toeplitz matrices there has been no proof of the superlinearity of any multilevel circulants. In this paper we show that such a proof is not possible since any multilevel circulant preconditioner is not superlinear, in the general case of multilevel Toeplitz matrices. Moreover, for matrices not necessarily Toeplitz, we present some general results proving that many popular structured preconditioners cannot be superlinear.

Any Circulant-Like Preconditioner for Multilevel Matrices Is Not Superlinear

How to recover missing data from an incomplete samples is a fundamental problem in mathematics and it has wide range of applications in image analysis and processing. Although many existing methods, e.g. various data smoothing methods and PDE approaches, are available in the literature, there is always a need to find new methods leading to the best solution according to various cost functionals. In this paper, we propose an iterative algorithm based on tight framelets for image recovery from incomplete observed data. The algorithm is motivated from our framelet algorithm used in high-resolution image reconstruction and it exploits the redundance in tight framelet systems. We prove the convergence of the algorithm and also give its convergence factor. Furthermore, we derive the minimization properties of the algorithm and explore the roles of the redundancy of tight framelet systems. As an illustration of the effectiveness of the algorithm, we give an application of it in impulse noise removal.

/pdf/convergence-analysis-of-tight-framelet-approach-for-missing-42x8z12919.pdf

Convergence analysis of tight framelet approach for missing data recovery

The authors extend some recent results of Di Fiore and Zellini [Linear Algebra Appl., to appear], obtaining new classes of formulas for the displacement operator-based decomposition of matrices. It is shown how an arbitrary matrix can be expressed as the sum of products of matrices belonging to matrix algebras associated with certain versions of sine and cosine transforms. Applications to the representation of the inverse of a Toeplitz and a Toeplitz plus 
Hankel matrix, with and without symmetry, are presented. Implications on the computation of the product of these matrices by a vector are discussed.

On the Use of Certain Matrix Algebras Associated with Discrete Trigonometric Transforms in Matrix Displacement Decomposition

Recently, a powerful two-phase method for restoring images corrupted with high level impulse noise has been developed. The main drawback of the method is the computational efficiency of the second phase which requires the minimization of a non-smooth objective functional. However, it was pointed out in (Chan et al. in Proc. ICIP 2005, pp. 125---128) that the non-smooth data-fitting term in the functional can be deleted since the restoration in the second phase is applied to noisy pixels only. In this paper, we study the analytic properties of the resulting new functional ?. We show that ?, which is defined in terms of edge-preserving potential functions ? ? , inherits many nice properties from ? ? , including the first and second order Lipschitz continuity, strong convexity, and positive definiteness of its Hessian. Moreover, we use these results to establish the convergence of optimization methods applied to ?. In particular, we prove the global convergence of some conjugate gradient-type methods and of a recently proposed low complexity quasi-Newton algorithm. Numerical experiments are given to illustrate the convergence and efficiency of the two methods.

/pdf/minimization-of-a-detail-preserving-regularization-3pmrsntgbn.pdf

Minimization of a Detail-Preserving Regularization Functional for Impulse Noise Removal

In this paper a new class of quasi-Newton methods, named ℒQN, is introduced in order to solve unconstrained minimization problems. The novel approach, which generalizes classical BFGS methods, is based on a Hessian updating formula involving an algebra ℒ of matrices simultaneously diagonalized by a fast unitary transform. The complexity per step of ℒQN methods is O(n log n), thereby improving considerably BFGS computational efficiency. Moreover, since ℒQN's iterative scheme utilizes single-indexed arrays, only O(n) memory allocations are required. Global convergence properties are investigated. In particular a global convergence result is obtained under suitable assumptions on f. Numerical experiences [7] confirm that ℒQN methods are particularly recommended for large scale problems.

/pdf/matrix-algebras-in-quasi-newton-methods-for-unconstrained-3wkvvwbpn6.pdf

Matrix algebras in Quasi-Newton methods for unconstrained minimization

Matrix Decompositions Using Displacement Rank and Classes of Commutative Matrix Algebras

https://www.mat.uniroma2.it/~tvmsscho/papers/DiFiore1.pdf

Carmine Di Fiore

Papers

On the Use of Certain Matrix Algebras Associated with Discrete Trigonometric Transforms in Matrix Displacement Decomposition

Minimization of a Detail-Preserving Regularization Functional for Impulse Noise Removal

Matrix algebras in Quasi-Newton methods for unconstrained minimization

Matrix Decompositions Using Displacement Rank and Classes of Commutative Matrix Algebras

Matrix algebras in optimal preconditioning