Showing papers on "Singular value decomposition published in 1983"

State-space and singular-value decomposition-based approximation methods for the harmonic retrieval problem

Abstract: We present new high-resolution methods for the problem of retrieving sinusoidal processes from noisy measurements. The approach taken is by use of the so-called principal-components method, which is a singular-value-decomposition-based approximate modeling method. The low-rank property and the algebraic structure of both the data matrix and the covariance matrix (under noise-free conditions) form the basis of exact modeling methods. In a noisy environment, however, the rank property is often perturbed, and singular-value decomposition is used to obtain a low-rank approximant in factored form. The underlying algebraic structure of these factors leads naturally to least-squares estimates of the state-space parameters of the sinusoidal process. This forms the basis of the Toeplitz approximation method, which offers a robust Pisarenko-like spectral estimate from the covariance sequence. Furthermore, the principle of Pisarenko’s method is extended to harmonic retrieval directly from time-series data, which leads to a direct-data approximation method. Our simulation results indicate that favorable resolution capability (compared with existing methods) can be achieved by the above methods. The application of these principles to two-dimensional signals is also discussed.

System Identification, Reduced-Order Filtering and Modeling via Canonical Variate Analysis

Abstract: Very general reduced order filtering and modeling problems are phased in terms of choosing a state based upon past information to optimally predict the future as measured by a quadratic prediction error criterion. The canonical variate method is extended to approximately solve this problem and give a near optimal reduced-order state space model. The approach is related to the Hankel norm approximation method. The central step in the computation involves a singular value decomposition which is numerically very accurate and stable. An application to reduced-order modeling of transfer functions for stream flow dynamics is given.

Computation of the Singular Value Decomposition Using Mesh-Connected Processors

Abstract: A cyclic Jacobi method for computing the singular value decomposition of an $mxn$ matrix $(m \geq n)$ using systolic arrays is proposed. The algorithm requires $O(n^{2})$ processors and $O(m + n \log n)$ units of time.

Limitation of the Inverse Problem in Body Surface Potential Mapping

Abstract: The inverse problem in electrocardiography is ill-conditioned, and small noise included in the measured potentials causes large errors in the solution. Since the inverse problem is mostly described as a linear problem, the entire problem has often been treated in terms of a transfer matrix. The degree of linear independence among the vectors in the transfer matrix, which is directly related to the stability of the solution, is well represented by the singular values of the transfer matrix. By means of the singular value decomposition of the transfer matrix, the stability of solution to the inverse problem has been discussed when the potential data contain noise or the transfer matrix includes some error. We have derived expressions of maximum possible error magnification and a root-mean-square error magnification and, in terms of these parameters, found that only 4 equivalent cardiac dipoles or only 15 independent epicardial potentials can be estimated from body surface potentials when they are measured with an accuracy as high as 99 percent.

Singular-value decomposition approach to time series modelling

Abstract: In various signal processing applications, as exemplified by spectral analysis, deconvolution and adaptive filtering, the parameters of a linear recursive model are to be selected so that the model is `most' representative of a given set of time series observations For many of these applications, the parameters are known to satisfy a theoretical recursive relationship involving the time series' autocorrelation lags Conceptually, one may then use this recursive relationship, with appropriate autocorrelation lag estimates substituted, to effect estimates for the operator's parameters A procedure for carrying out this parameter estimation is given which makes use of the singular-value decomposition (SVD) of an extended-order autocorrelation matrix associated with the given time series Unlike other SVD modelling methods, however, the approach developed does not require a full-order SVD determination Only a small subset of the matrix's singular values and associated characteristic vectors need be computed This feature can significantly alleviate an otherwise overwhelming computational burden that is necessitated when generating a full-order SVD Furthermore, the modelling performance of this new method has been found empirically to excel that of a near maximum-likelihood SVD method as well as several other more traditional modelling methods

A Lanczos Algorithm for Computing Singular Values and Vectors of Large Matrices

Abstract: Any real matrix A has associated with it the real symmetric matrix \[ B \equiv \left(\begin{array}{*{20}c} 0 \\ {A^T } \\ \end{array} \begin{array}{*{20}c} A \\ 0 \\ \end{array} \right) \] whose positive eigenvalues are the nonzero singular values of A. Using B and our Lanczos algorithms for computing eigenvalues and eigenvectors of very large real symmetric matrices, we obtain an algorithm for computing singular values and singular vectors of large sparse real matrices. This algorithm provides a means for computing the largest and the smallest or even all of the distinct singular values of many matrices

Realization algorithms and approximation methods of bilinear systems

Abstract: High-dimensional mathematical models of bilinear control systems are often not amenable due to the difficulty in implementation. In this paper, we address the problem of order-reduction for both discrete and continuous time bilinear systems. Two model-reduction algorithms are presented; one is based on the singular value decomposition of the generalized Hankel matrix (the Hankel Approach) and the other is based on the eigenvalue / eigenvector decomposition of the product of reachability and observability Gramians (the Gramian Approach). Equivalence between these two algorithms is established. The main result of this paper is a systematic approach for obtaining reduced-order bilinear models. Furthermore, the bilinear reachabiiity and observability Gramians are shown to be obtainable from the solutions of generalized Lyapunov equations. Computer simulations of a neutron-kinetic system are presented to illustrate the effectiveness of the proposed model-reduction algorithms.

A Review Of Signal Processing With Systolic Arrays

Abstract: This paper reviews recent developments in signal processing and surveys recent progress in parallel processing algorithms and architectures for their real-time implementation. It has previously been shown1-2 that the major computational requirements for many important real-time signal processing tasks can be reduced to a common set of basic matrix operations including matrix-vector multiplication, matrix-matrix multiplication and addition, matrix inversion, solution of systems of linear equations, least squares approximate solution of linear systems, eigensystem solution, generalized eigen-systems solution, and singular value decomposition (SVD) of matrices. To this list, we would now add the generalized singular value decompositions of Van Loan3,4 and Paige-Saunders5. The first five matrix operations listed above may be computed non-iteratively, and systolic array architectures and algorithms are available which provide modular parallelism, local interconnects, regular data flow, and high efficiency, with the efficiency essentially constant as the parallelism is increased6-8. Parallel computation of eigensystems, generalized eigensystems, the singular value decomposition, and the generalized singular value decomposition is more difficult, since the computation is necessarily iterative, and it is difficult to utilize only local communication between processing elements while maintaining high efficiency. Algorithms for the latter problems are therefore still the subject of intensive research.© (1983) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Visualization of Matrix Singular Value Decomposition

Abstract: (1983). Visualization of Matrix Singular Value Decomposition. Mathematics Magazine: Vol. 56, No. 3, pp. 161-167.

Modal decomposition signal subspace algorithms

Abstract: The broadband emitter location problem is treated as a separable multidimensional spectrum estimation problem by modal decomposition of the estimated spectral density matrix at the poles of the sources. This modal decomposition method extends conventional sinusoidal (frequency bin or harmonic retrieval type) separation approaches to broadband sources. The modal decomposition is accomplished using overdetermined ARMA pole estimates and a Prony-like method with singular value decomposition for residue estimation. The signal subspace approach of Schmidt (1979), Su and Morf (1982) is extended and applied to the decomposed spectral residues for determination of source locations. Simulation results are given with comparison to MLM, MUSIC and EV methods applied at source peaks.

The Relationship Between Word And Bit Level Systolic Arrays As Applied To Matrix X Matrix Multiplication

Abstract: Westglen Engineering Ltd., 4 Mercia Way,Bell's Close Industrial Estate,Newcastle upon Tyne NE15 6UF, EnglandAbstractThe mapping of matrix x matrix multiplication on to both word and bit level systolicarrays has been investigated. It has been found that well defined word and bit level dataflow constraints must be satisified within such circuits. An efficient and highly regularbit level array has been generated by exploiting the basic compatibilities in data flowsymmetries at each level of the problem. A detailed description of the circuit which emergesis given and some details relating to its practical implementation are discussed.IntroductionConsiderable progress has been achieved in recent years in the development of algorithmsand architectures which exploit the potential computational power of VLSI. In particular,the systolic array approach [1] is gaining increasing popularity and has now been applied toa wide range of problems. Most of the effort has been concentrated at the word or systemlevel and this has produced computational structures for a number of problems in linearalgebra. These include arrays for the solution of linear equations [2], least squaresproblems [3]and singular value decomposition [4]. The work which we have undertaken hasbeen concentrated mainly at the other end of the spectrum and we have recognised that thesystolic array approach applied at the bit level provides one with an extremely powerfulapproach to VLSI chip design. In particular, we have shown that many important signal anddata processing functions can be implemented using highly repetitive patterns of simple bitlevel cells having little or no long range connectivity [5 -7] and some of these ideas havesince been implemented as integrated circuits [8,9]. The basis of our approach is that wetreat problems from the outset at the bit level and usually no subset of cells within theresulting arrays can be associated with a specific multiplication or addition at word level.Given the attraction of the bit level approach (for example, the ability to completely tilea plane of silicon with simple cells) the,question arises as to how one can best map a givenword level problem on to a bit level array. In this paper we investigate this matter withinthe context of matrix x matrix multiplication and demonstrate for this example how maintain-ing similar data flow geometries at different levels of the problem produces an efficientand highly regular bit level array.The organisation of this paper is as follows. In section 2 we break matrix x matrixmultiplication down from sums of word level products to sums of bit level products and thendiscuss the data flow constraints at each level. Details of the structure which emerges arethen described in section 3, where we also discuss the envisaged implementation of thecircuit. The important conclusions which can be drawn from the work are given in section 4.Analysis of problem at word and bit levelThe multiplication of two n x n matrices A = (aik) and B = (bk.) to form a matrix productC = (cij) is defined bycii = aik bkj

The Singular Value Decomposition in Product Form

Abstract: A gain of about $50\% $ in the CP-time required for the calculation of the singular value decomposition of a general matrix can be achieved by not forming the orthogonal factors explicitly, but storing the Householder reflections and Jacobi rotations that compose them. An efficient method for storing the Jacobi rotations is given. The storage required for the resulting decomposition is, for general matrices, about $1\frac{1} {2}$ times what is usual, but it is not larger than usual for matrices arising in ill-posed problems. The storage required for the decomposition of a general matrix can be reduced to what is usual by giving up part of the gain in efficiency and applying an “ultimate shift” strategy.

A Systolic Architecture for Singular Value Decomposition

Abstract: : Systolic arrays are highly parallel computing structures specific to particular computing tasks. They are well-suited for reliable and inexpensive implementation using many identical VLSI components. The designs consist of one and two-dimensional lattices of identical processing elements. Communication of data occurs only between neighboring cells. Control signals propagate through the array like data. These characteristics make it feasible to construct very large arrays. Several modern methods in digital signal processing require real time solution of some of the basic problems of linear algebra. Fortunately systolic arrays have been developed for many of these problems. But several gaps remain. Only partially satisfying results have been obtained for the eigenvalue and singular value decompositions, for example. This document considers a systolic array for the singular value decomposition (SVD). In this paper the author discusses two topics. First, he shows how an architecture for computing the eigenvalues of a symmetric matrix can be modified to compute singular values and vectors. Second, he discusses the implementation using VLSI chips of these systolic eigenvalue and SVD arrays.

Accuracy and Resolution of Earth Radiation Budget Estimates

Abstract: A numerical filter inversion technique that reduces wide-angle satellite measurements to top-of-the-atmosphere radiant exitances has been proposed for the Earth Radiation Budget Experiment (ERBE). The matrix formulation of this technique is presented, and the design of the numerical filter is discussed. The filter is smoothed with a singular value decomposition. The inversion process is simulated by generating synthetic measurements from a 24 degree spherical harmonic radiation field derived from Nimbus 6 ERB data. The numerical filter is applied to these measurements after they are corrupted with instrument error. The results are curves of expected error versus resolution area.

A generalized SVD analysis of some weighting methods for equality constrained least squares

Abstract: The method of weighting is a useful way to solve least squares problems that have linear equality constraints. New error bounds for the method are derived using the generalized singular value decomposition. The analysis clarifies when the weighting approach is successful and suggests modifications when it is not.

Singular Value Computations with Systolic Arrays.

Abstract: : Systolic arrays are constructed for bandwidth reduction and singular value decomposition of m x n matrices, wlog, m or = n. The underlying algorithms are unconditionally stable. Since input and output occurs, as in previous designs, by diagonals arrays can be directly appended to further reduce the computation time. Consequently, the designs will be most efficient for matrices with a fairly small and dense band.

Direct design of multivariable control systems through singular value decomposition

Abstract: Recent research in controller design through nonlinear programming and singular value decomposition has produced more direct methodologies for designing mu1 tivariable controllers to meet engineering requirements. A direct relationship between singular values of the return difference matrix and multivariable phase and gain margins has been recently described. Also, system response characteristics can be directly related to the closed-loop eigenvalues. Both stability and response measures can be combined into a s ingle criterion function which is then minimized by a nonlinear programming algorithm. The method is illustrated through design of the pitch plane autopilot for a 10 ft wingspan drojqe aircraft.

Notes on economic time series analysis : system theoretic perspectives

Abstract: 1 Introduction.- 2 The Notion of State.- 3 Time-invariant Linear Dynamics.- 3.1 Continuous time systems.- 3.2 Inverse systems.- 3.3 Discrete-time sequences.- 4 Time Series Representation.- 5 Equivalence of ARMA and State Space Models.- 5.1 AR models.- 5.2 MA models.- 5.3 ARMA models.- Examples.- 6 Decomposition of Data into Cyclical and Growth Components.- 6.1 Reference paths and variational dynamic models.- 6.2 Log-linear models as variational models.- 7 Prediction of Time Series.- 7.1 Prediction space.- 7.2 Equivalence.- 7.3 Cholesky decomposition and innovations.- 8 Spectrum and Covariances.- 8.1 Covariance and spectrum.- 8.2 Spectral factorization.- 8.3 Computational aspects.- Sample covariance Matrices.- Example.- 9 Estimation of System Matrices: Initial Phase.- 9.1 System matrices.- 9.2 Approximate model.- 9.3 Rank determination of Hankel matrices: singular value decomposition theorem.- 9.4 Internally balanced model.- example.- construction..- properties of internally balanced models.- principal component analysis.- 9.5 Inference about the model order.- 9.6 Choices of basis vectors.- 9.7 State space model.- example.- 9.8 ARMA (input-output) model.- 9.9 Canonical correlation.- 10 Innovation Processes.- 10.1 Orthogonal projection.- 10.2 Kaiman filters.- 10.3 Innovation model.- causal invertibility.- 10.4 Output statistics Kaiman filter.- 10.5 Spectral factorization.- 11 Time Series from Intertemporal Optimization.- 11.1 Example: dynamic resource allocation problem.- 11.2 Quadratic regulation problems.- discrete-time systems.- 11.3 Parametric analysis of optimal solutions.- choice of weighting matrices.- 12 Identification.- 12.1 Closed-loop systems.- 12.2 Identifiability of a closed-loop system.- 13 Time Series from Rational Expectations Models 140.- 13.1 Moving Average processes.- 13.2 Autoregressive processes.- 13.3 ARMA models.- 13.4 Examples.- example.- example.- example.- case of common information pattern.- case of differential information set.- 14 Numerical Examples.- Mathematical Appendices.- A.1 Solutions of difference equations.- A.2 Geometry of weakly stationary stochastic sequences.- A.3 Principal components.- A.4 Fourier transforms.- A.5 The z-transform.- A.6 Some useful relations for quadratic forms.- A.7 Calculation of the inverse, (z I-A)-1.- A.8 Sensitivity analysis of optimal solutions: scalar-valued case.- A.9 Common factor in ARMA models and controllability.- A.10 Non-controllability and singular probability distribution.- A.11 Spectral decomposition representation.- A.12 Singular value decomposition theorem.- A.13 Hankel matrices.- A.14 Dual relations.- A.15 Quadratic regulation problem: continuous time systems.- A.16 Maximum principle: discrete-time dynamics.- A.17 Policy reaction functions, stabilization policy and modes.- A.18 Dynamic policy multipliers.- References.

Numerical aspects of the inverse function theory

Abstract: The state-estimation problem can be formulated as an inversion problem From this point of view, we first review known results about the relationship between the existence of an unbiased estimate and the observability condition of a nonlinear system The inversion relationship between the Fisher information matrix and the error covariance matrix, evaluated at the true state of an observable nonlinear System, is extended to an unobservable nonlinear system A singular value decomposition technique solves the sensitivity question of a nonlinear inversion problem The technique is applied to evaluate a radar calibration procedure

Linear or square array for eigenvalue and singular value decompositions

Abstract: : For many signal and image processing applications, such as high resolution spectral estimation, image data compression etc., eigenvalue and singular value decompositions have emerged as extremely powerful and efficient computational tools. As far as the symmetric eigenvalue problem is concerned, QL and QR algorithms ... have emerged as the most effective way of finding all the eigenvalues of a small symmetric matrix. A full matrix is first reduced to tridiagonal form by a sequence of reflections and then the QL (QR) algorithm swiftly reduces the off diagonal elements until they are negligible. The algorithm repeatedly applies a complicated similarity transformation to the result of the previous transformation, thereby producing a sequence of matrices that converges to a diagonal form. What is more, the tridiagonal form is preserved. Therefore, the QR algorithm can be regarded as the best sequential algorithm available to date. The question is whether or not the QR algorithm may retain that same effectiveness when mapped into a parallel algorithm on a square or linear multiprocessor array. In this note, the authors offer an answer to this question using the computational wavefront notion.

Restoration of noisy images using a raised cosine function approximation

Abstract: In order to implement image restoration techniques on a digital computer an accurate representation of the image in discrete form is required. A new set of functions called raised cosine functions is suggested for this purpose. These functions have properties similar to those of the spline functions which are useful for image representation and inversion of the degrading phenomenon. The singular value decomposition in the raised cosine function domain is used for restoration of noisy degraded images.

Time series modeling via general linear estimation theory

Abstract: A procedure for time series modeling is presented which combines a general linear estimation approach with a smoothing singular value decomposition operation. The linear estimator is allowed to possess both causal and anticausal terms. This structure is found to yield better performance capabilities than strictly causal or anticausal structures. Upon using this less restrictive linear estimator with the smoothing properties of a singular value decomposition operation, a time series modeling procedure with superresolution capabilities in low signal-to-noise environments is evolved. The optimality of this approach is analytically established for the important case of two closely spaced (in frequency) sinusoids in white noise.

Model Reduction of Bilinear Control Systems

Abstract: In this paper, the problem of model reduction for discrete bilinear control systems is considered. Based upon the fundamental relationships among the system reachability, observability and stability, two methods for obtaining the reduced-order models are introduced. The first method applies the singular value decomposition on the generalized Hankel matrix which is defined in terms of the impulse response data. The second method utilizes the factorization of the reachability and observability Gramians. These Gramians are shown to satisfy generalized Lyapunov matrix equations under certain condtions. The order-reduction algorithms developed for bilinear systems have been tested on a fifth-order neutron kinetic model. A third-order model which is balanced with respect to both reachability and observability is shown to accurately approximate the original fifth-order model.

Matrix pencils : proceedings of a conference held at Pite Havsbad, Sweden, March 22-24, 1982

Abstract: The condition number of equivalence transformations that block diagonalize matrix pencils.- An approach to solving the spectral problem of A-?B.- On computing the Kronecker canonical form of regular (A-?B)-pencils.- Reducing subspaces: Definitions, properties and algorithms.- Differential/algebraic systems and matrix pencils.- Approximation of eigenvalues defined by ordinary differential equations with the Tau method.- The two-sided arnoldi algorithm for nonsymmetric eigenvalue problems.- Projection methods for solving large sparse eigenvalue problems.- The generalized eigenvalue problem in shipdesign and offshore industry - a comparison of traditional methods with the lanczos process.- On the practical use of the lanczos algorithm in finite element applications to vibration and bifurcation problems.- Implementation and applications of the spectral transformation lanczos algorithm.- Preconditioned iterative methods for the generalized eigenvalue problem.- On bounds for symmetric eigenvalue problems.- A method for computing the generalized singular value decomposition.- Perturbation analysis for the generalized eigenvalue and the generalized singular value problem.- A generalized SVD analysis of some weighting methods for equality constrained least squares.- On angles between subspaces of a finite dimensional inner product space.- The multivariate calibration problem in chemistry solved by the PLS method.

Optical Singular Value Decomposition For The Ax = b Problem

Abstract: Optical approaches to solving the Ax = b problem have suffered from four difficulties: (1) an inability to handle the problem for nonsquare A, (2) the necessity of insuring convergence for nonsingular A, (3) the inability to handle a singular A, and (4) inaccuracies due to an ill conditioned A. We show that these problems can all be solved or mitigated by singular value decomposition (SVD). An accurate approach to optical SVD is shown.© (1983) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.