The ubiquitous Kronecker product

We present an efficient algorithm to solve a class of two- and 25-dimensional (2-D and 25-D) Fredholm integrals of the first kind with a tensor product structure and nonnegativity constraint on the estimated parameters of interest in an optimization framework A zeroth-order regularization functional is used to incorporate a priori information about the smoothness of the parameters into the problem formulation We adapt the Butler-Reeds-Dawson (1981) algorithm to solve this optimization problem in three steps In the first step, the data are compressed using singular value decomposition (SVD) of the kernels The tensor-product structure of the kernel is exploited so that the compressed data is typically a thousand fold smaller than the original data This size reduction is crucial for fast optimization In the second step, the constrained optimization problem is transformed to an unconstrained optimization problem in the compressed data space In the third step, a suboptimal value of the smoothing parameter is chosen by the BRD method Steps 2 and 3 are iterated until convergence of the algorithm We demonstrate the performance of the algorithm on simulated data

Solving Fredholm integrals of the first kind with tensor product structure in 2 and 2.5 dimensions

In iterative image restoration methods, implementation of efficient matrix vector multiplication, and linear system solves for preconditioners, can be a tedious and time consuming process. Different blurring functions and boundary conditions often require implementing different data structures and algorithms. A complex set of computational methods is needed, each likely having different input parameters and calling sequences. This paper describes a set of Matlab tools that hide these complicated implementation details. Combining the powerful scientific computing and graphics capabilities in Matlab, with the ability to do object-oriented programming and operator overloading, results in a set of classes that is easy to use, and easily extensible.

Iterative Methods for Image Deblurring: A Matlab Object-Oriented Approach

SUMMARY Statistical modeling of space-time data has often been based on separable covariance functions, that is, covariances that can be written as a product of a purely spatial covariance and a purely temporal covariance. The main reason is that the structure of separable covariances dramatically reduces the number of parameters in the covariance matrix and thus facilitates computational procedures for large space-time data sets. In this paper, we discuss separable approximations of nonseparable space-time covariance matrices. Specifically, we describe the nearest Kronecker product approximation, in the Frobenius norm, of a space-time covariance matrix. The algorithm is simple to implement and the solution preserves properties of the space-time covariance matrix, such as symmetry, positive definiteness, and other structures. The separable approximation allows for fast kriging of large space-time data sets. We present several illustrative examples based on an application to data of Irish wind speeds, showing that only small differences in prediction error arise while computational savings for large data sets can be obtained. Copyright © 2007 John Wiley & Sons, Ltd.

Separable approximations of space‐time covariance matrices

Lanczos-hybrid regularization methods have been proposed as effective approaches for solving largescale ill-posed inverse problems. Lanczos methods restrict the solution to lie in a Krylov subspace, but they are hindered by semi-convergence behavior, in that the quality of the solution first increases and then decreases. Hybrid methods apply a standard regularization technique, such as Tikhonov regularization, to the projected problem at each iteration. Thus, regularization in hybrid methods is achieved both by Krylov filtering and by appropriate choice of a regularization parameter at each iteration. In this paper we describe a weighted generalized cross validation (WGCV) method for choosing the parameter. Using this method we demonstrate that the semi-convergence behavior of the Lanczos method can be overcome, making the solution less sensitive to the number of iterations.

/pdf/a-weighted-gcv-method-for-lanczos-hybrid-regularization-56z5tklocu.pdf

A weighted-GCV method for Lanczos-hybrid regularization.

Kronecker product and svd approximations in image restoration

This paper considers the problem of finding n × n matrices Ak and Bk that minimize $||T - \sum A_k \otimes B_k||_F$, where $\otimes$ denotes Kronecker product and T is a banded n × n block Toeplitz matrix with banded n × n Toeplitz blocks. It is shown that the optimal Ak and Bk are banded Toeplitz matrices, and an efficient algorithm for computing the approximation is provided. An image restoration problem from the Hubble Space Telescope (HST) is used to illustrate the effectiveness of an approximate SVD preconditioner constructed from the Kronecker product decomposition.

Optimal Kronecker Product Approximation of Block Toeplitz Matrices

A Newton method to solve total least squares problems for Toeplitz systems of equations is considered. When coupled with a bisection scheme, which is based on an efficient algorithm for factoring Toeplitz matrices, global convergence can be guaranteed. Circulant and approximate factorization preconditioners are proposed to speed convergence when a conjugate gradient method is used to solve linear systems arising during the Newton iterations.

A total least squares method for Toeplitz systems of equations

In image restoration, a separable, spatially variant blurring function has the form k(x, y; s, 1) =ki(x,s)k2(y, t). If this kernel is known, then discretizations lead to a blurring matrix which is a Kronecker product of two matrices of smaller dimension. If k is not known precisely, such a discretization is not possible. In this paper we describe an interpolation scheme to construct a Kronecker product approximation to the blurring matrix from a set of observed point spread functions for separable, or nearly separable, spatially variant blurs. An approximate singular value decomposition is then computed from this Kronecker factorization.
 
Keywords: Image restoration, Interpolation, Kronecker product, space variant blur, SVD

Julie Kamm

Papers

Kronecker product and svd approximations in image restoration

Optimal Kronecker Product Approximation of Block Toeplitz Matrices

A total least squares method for Toeplitz systems of equations

Kronecker product and SVD approximations for separable spatially variant blurs