Convergence criteria for iterative restoration methods
01 Feb 1983-IEEE Transactions on Acoustics, Speech, and Signal Processing (IEEE)-Vol. 31, Iss: 1, pp 129-136
TL;DR: The use of the residual signal in defining a practical criterion to indicate when the numerical algorithm has converged is investigated and the advantage of this criterion over the criterion of examining successive iterations is demonstrated.
Abstract: While many iterative signal restoration methods have been shown to converge in the mathematical sense, a practical criterion is needed to indicate when the numerical algorithm has converged. The use of the residual signal in defining such a criterion is investigated. The advantage of this criterion over the criterion of examining successive iterations is demonstrated.
Citations
More filters
01 Feb 1993
TL;DR: The author synthesizes a single, general framework from various approaches to set theoretic estimation, examines its fundamental philosophy, goals, and analytical techniques, and relates it to conventional methods.
Abstract: Explains set theoretic estimation, which is governed by the notion of feasibility and produces solutions whose sole property is to be consistent with all information arising from the observed data and a priori knowledge. Each piece of information is associated with a set in the solution space, and the intersection of these sets, the feasibility set, represents the acceptable solutions. The practical use of the set theoretic framework stems from the existence of efficient techniques for finding these solutions. Many scattered problems in systems science and signal processing have been approached in set theoretic terms over the past three decades. The author synthesizes a single, general framework from these various approaches, examines its fundamental philosophy, goals, and analytical techniques, and relates it to conventional methods. >
666 citations
TL;DR: An error analysis based on an objective mean-square-error (MSE) criterion is used to motivate regularization and two approaches for choosing the regularization parameter and estimating the noise variance are proposed.
Abstract: The application of regularization to ill-conditioned problems necessitates the choice of a regularization parameter which trades fidelity to the data with smoothness of the solution. The value of the regularization parameter depends on the variance of the noise in the data. The problem of choosing the regularization parameter and estimating the noise variance in image restoration is examined. An error analysis based on an objective mean-square-error (MSE) criterion is used to motivate regularization. Two approaches for choosing the regularization parameter and estimating the noise variance are proposed. The proposed and existing methods are compared and their relationship to linear minimum-mean-square-error filtering is examined. Experiments are presented that verify the theoretical results. >
551 citations
Cites background from "Convergence criteria for iterative ..."
...is Chi-square distributed with variance o2 and M2 degrees of freedom [ 33 ], [34]....
[...]
01 May 1990
TL;DR: In this paper, the authors discuss the use of iterative restoration algorithms for the removal of linear blurs from photographic images that may also be degraded by pointwise nonlinearities such as film saturation and additive noise.
Abstract: The authors discuss the use of iterative restoration algorithms for the removal of linear blurs from photographic images that may also be assumed to be degraded by pointwise nonlinearities such as film saturation and additive noise. Iterative algorithms allow for the incorporation of various types of prior knowledge about the class of feasible solutions, can be used to remove nonstationary blurs, and are fairly robust with respect to errors in the approximation of the blurring operator. Special attention is given to the problem of convergence of the algorithms, and classical solutions such as inverse filters, Wiener filters, and constrained least-squares filters are shown to be limiting solutions of variations of the iterations. Regularization is introduced as a means for preventing the excessive noise magnification that is typically associated with ill-conditioned inverse problems such as the deblurring problem, and it is shown that noise effects can be minimized by terminating the algorithms after a finite number of iterations. The role and choice of constraints on the class of feasible solutions are also discussed. >
513 citations
TL;DR: The recovery criterion defines the class of images that are acceptable as solutions to the problem and the recovery method is a numerical algorithm that will produce a solution to the recovery problem, that is, an image that satisfies the recovery criterion as discussed by the authors.
Abstract: Publisher Summary Image recovery is a broad discipline that encompasses the large body of inverse problems, in which an image h is to be inferred from the observation of data x consisting of signals physically or mathematically related to it. Image restoration and image reconstruction are the two main sub-branches of image recovery. The term “image restoration” usually applies to the problem of estimating the original form h of a degraded image x . The following four basic elements are required to solve an image recovery problem: (1) a data formation model, (2) a priori information, (3) a recovery criterion, and (4) a solution method. The recovery criterion defines the class of images that are acceptable as solutions to the problem. It is chosen by the user on grounds that may include experience, compatibility with the available a priori knowledge, personal convictions on the best way to solve the problem, and ease of implementation. The traditional approach has been to use a criterion of optimality, which usually leads to a single best solution. An alternative approach is to use a criterion of feasibility, in which consistency with all prior information and the data defines a set of equally acceptable solutions. The solution method is a numerical algorithm that will produce a solution to the recovery problem—that is, an image that satisfies the recovery criterion. Modification can be made in two directions: in the conventional image recovery framework, one seeks to preserve the notion of an optimal solution, whereas in the set theoretic framework the emphasis is placed on feasibility.
472 citations
TL;DR: In this paper, approximate expressions for the mean and variance of implicitly defined estimators of unconstrained continuous parameters are derived using the implicit function theorem, the Taylor expansion, and the chain rule.
Abstract: Many estimators in signal processing problems are defined implicitly as the maximum of some objective function. Examples of implicitly defined estimators include maximum likelihood, penalized likelihood, maximum a posteriori, and nonlinear least squares estimation. For such estimators, exact analytical expressions for the mean and variance are usually unavailable. Therefore, investigators usually resort to numerical simulations to examine the properties of the mean and variance of such estimators. This paper describes approximate expressions for the mean and variance of implicitly defined estimators of unconstrained continuous parameters. We derive the approximations using the implicit function theorem, the Taylor expansion, and the chain rule. The expressions are defined solely in terms of the partial derivatives of whatever objective function one uses for estimation. As illustrations, we demonstrate that the approximations work well in two tomographic imaging applications with Poisson statistics. We also describe a "plug-in" approximation that provides a remarkably accurate estimate of variability even from a single noisy Poisson sinogram measurement. The approximations should be useful in a wide range of estimation problems.
426 citations
References
More filters
TL;DR: It is demonstrated that, for convolution-type models of image restoration, special properties of the linear system of equations can be used to reduce the computational requirements.
Abstract: Constrained least squares estimation is a technique for solution of integral equations of the first kind The problem of image restoration requires the solution of an integral equation of the first kind However, application of constrained least squares estimation to image restoration requires the solution of extremely large linear systems of equations In this paper we demonstrate that, for convolution-type models of image restoration, special properties of the linear system of equations can be used to reduce the computational requirements The necessary computations can be carried out by the fast Fourier transform, and the constrained least squares estimate can be constructed in the discrete frequency domain A practical procedure for constrained least squares estimation is presented, and two examples are shown as output from a program for the CDC 7600 computer which performs the constrained least squares restoration of digital images
590 citations
01 Apr 1981
TL;DR: It is shown that by predistorting the signal (and later removing this predistortion) it is possible to achieve spectral extrapolation, to broaden the class of signals for which these algorithms achieve convergence, and to improve their performance in the presence of broad-band noise.
Abstract: This paper describes a rather broad class of iterative signal restoration techniques which can be applied to remove the effects of many different types of distortions. These techniques also allow for the incorporation of prior knowledge of the signal in terms of the specification of a constraint operator. Conditions for convergence of the iteration under various combinations of distortions and constraints are explored. Particular attention is given to the use of iterative restoration techniques for constrained deconvolution, when the distortion band-limits the signal and spectral extrapolation must be performed. It is shown that by predistorting the signal (and later removing this predistortion) it is possible to achieve spectral extrapolation, to broaden the class of signals for which these algorithms achieve convergence, and to improve their performance in the presence of broad-band noise.
465 citations
TL;DR: In this article, a set of conditions under which a sequence is uniquely specified by the phase or samples of the phase of its Fourier transform was developed. But these conditions are distinctly different from the minimum or maximum phase conditions, and are applicable to both one-dimensional and multidimensional sequences.
Abstract: In this paper, we develop a set of conditions under which a sequence is uniquely specified by the phase or samples of the phase of its Fourier transform, and a similar set of conditions under which a sequence is uniquely specified by the magnitude of its Fourier transform. These conditions are distinctly different from the minimum or maximum phase conditions, and are applicable to both one-dimensional and multidimensional sequences. Under the specified conditions, we also develop several algorithms which may be used to reconstruct a sequence from its phase or magnitude.
439 citations
TL;DR: In this article, singular value decomposition (SVD) and pseudoinverse techniques are used for image restoration in space-variant point spread functions (SVPSF).
Abstract: The use of singular value decomposition (SVD) techniques in digital image processing is of considerable interest for those facilities with large computing power and stringent imaging requirements. The SVD methods are useful for image as well as quite general point spread function (impulse response) representations. The methods represent simple extensions of the theory of linear filtering. Image enhancement examples will be developed illustrating these principles. The most interesting cases of image restoration are those which involve space variant imaging systems. The SVD, combined with pseudoinverse techniques, provides insight into these types of restorations. Illustrations of large scale N2× N2point spread function matrix representations are discussed along with separable space variant N2× N2point spread function matrix examples. Finally, analysis and methods for obtaining a pseudoinverse of separable space variant point spread functions (SVPSF's) are presented with a variety of object and imaging system dagradations.
362 citations
TL;DR: In this article, the authors studied the convergence properties of the first kind of the Fredholm integral equation with respect to singular functions of K. The general theory of singular functions is interpreted and extended for the study of the iteration, and a quantitative method of choosing D to shape the response to singular function of K is derived.
Abstract: For the Fredholm integral equation of the first kind, written notationally as $Kf = g$, $g \in L_2 [0,1]$, we study the behavior of the iteration \[ \hat f_k = \hat f_{k - 1} + DK^ * (g - K\hat f_{k - 1} ),\quad k = 1,2, \cdots ,\] both with respect to its convergence properties and its response to singular functions of K. Here $K^ * $ is the adjoint of K, $\hat f_0 $ is a suitable starting function and D is a fixed linear operator to be chosen. The general theory of singular functions is interpreted and extended for the study of the iteration, and a quantitative method of choosing D to shape the response to singular functions of K is derived. Several specific matrix and integral-equation examples are presented.
162 citations