Showing papers on "Singular value decomposition published in 1992"
TL;DR: In this paper, the authors introduce a conceptual framework for comparing methods that isolate important coupled modes of variability between time series of two fields, including principal component analysis with the fields combined (CPCA), canonical correlation analysis (CCA), and singular value decomposition of the covariance matrix between the two fields (SVD).
Abstract: This paper introduces a conceptual framework for comparing methods that isolate important coupled modes of variability between time series of two fields. Four specific methods are compared: principal component analysis with the fields combined (CPCA), canonical correlation analysis (CCA) and a variant of CCA proposed by Barnett and Preisendorfer (BP), principal component analysis of one single field followed by correlation of its component amplitudes with the second field (SFPCA), and singular value decomposition of the covariance matrix between the two fields (SVD). SVD and CPCA are easier to implement than BP, and do not involve user-chosen parameters. All methods are applied to a simple but geophysically relevant model system and their ability to detect a coupled signal is compared as parameters such as the number of points in each field, the number of samples in the time domain, and the signal-to-noise ratio are varied. In datasets involving geophysical fields, the number of sampling times ma...
1,482 citations
TL;DR: In this paper, single field principal component analysis (PCA), direct singular value decomposition (SVD), canonical correlation analysis (CCA), and combined PCA of two fields are applied to a 39-winter dataset consisting of normalized seasonal mean sea surface temperature anomalies over the North Pacific and concurrent 500-mb height anomaly over the same region.
Abstract: Single field principal component analysis (PCA), direct singular value decomposition (SVD), canonical correlation analysis (CCA), and combined principal component analysis (CPCA) of two fields are applied to a 39-winter dataset consisting of normalized seasonal mean sea surface temperature anomalies over the North Pacific and concurrent 500-mb height anomalies over the same region. The CCA solutions are obtained by linear transformations of the SVD solutions. Spatial patterns and various measures of the variances and covariances explained by the modes derived from the different types of expansions are compared, with emphasis on the relative merits of SVD versus CCA. Results for two different analysis domains (i.e., the Pacific sector versus a full hemispheric domain for the 500-mb height field) are also compared in order to assess the domain dependence of the two techniques. The SVD solution is also compared with the results of 28 Monte Carlo simulations in which the temporal order of the SST gri...
619 citations
TL;DR: The chapter describes the way in which the singular value decomposition of a noise-free data set for which the spectra, f, and concentration, c, vectors are known can be calculated from consideration of the integrated overlaps of these components.
Abstract: Publisher Summary This chapter summarizes the properties of the singular value decomposition, which are relevant for data analysis. The chapter describes the way in which the singular value decomposition (SVD) of a noise-free data set for which the spectra, f , and concentration, c , vectors are known can be calculated from consideration of the integrated overlaps of these components. Because data analysis necessarily begins with matrices, which are “noisy” at some level of precision, the chapter describes some of the properties of the SVD of matrices, which contain noise. It describes the SVD of random matrices (that is, matrices containing only noise). The chapter explores the way the random amplitudes are distributed in the SVD output when noise is added to a data matrix by using perturbation theory. The chapter discusses the significant advantages that result from the application of SVD and complementary processing techniques for the analysis of large sets of experimental data.
550 citations
TL;DR: In this paper, the authors survey several numerical tools that can be used for the analysis and solution of systems of linear algebraic equations derived from Fredholm integral equations of the first kind.
Abstract: The author surveys several numerical tools that can be used for the analysis and solution of systems of linear algebraic equations derived from Fredholm integral equations of the first kind. These tools are based on the singular value decomposition (SVD) and the generalized SVD, and they allow the user to study many details of the integral equation. The tools also aid the user in choosing a good regularization parameter that balances the influence of regularization and perturbation errors.
450 citations
01 Apr 1992
TL;DR: Four numerical methods for computing the singular value decomposition (SVD) of large sparse matrices on a multiprocessor architecture are presented and may help advance the development of future out-of-core sparse SVD methods, which can be used to handle extremely large sparsematrices associated with extremely large databases in query-based information-retrieval applications.
Abstract: We present four numerical methods for computing the singular value decomposition SVD of large sparse matrices on a multiprocessor architecture. We emphasize Lanczos and subspace iteration-based methods for determining several of the largest singular triplets singular values and corresponding left- and right-singular vectors for sparse matrices arising from two practical applications: information retrieval and seismic reflection tomography. The target architectures for our implementations are the CRAY-2S/4-128 and Alliant FX/80. The sparse SVD problem is well motivated by recent information-retrieval techniques in which dominant singular values and their corresponding singular vectors of large sparse term-document matrices are desired, and by nonlinear inverse problems from seismic tomography applications which require approximate pseudo-inverses of large sparse Jacobian matrices. This research may help advance the development of future out-of-core sparse SVD methods, which can be used, for example, to handle extremely large sparse matrices 0 ? 106 rows or columns associated with extremely large databases in query-based information-retrieval applications.
403 citations
TL;DR: It is shown that a different decompositions, called the URV decomposition, is equally effective in exhibiting the null space and can be updated in O(p/sup 2/) time.
Abstract: In certain signal processing applications it is required to compute the null space of a matrix whose rows are samples of a signal with p components. The usual tool for doing this is the singular value decomposition. However, the singular value decomposition has the drawback that it requires O(p/sup 3/) operations to recompute when a new sample arrives. It is shown that a different decomposition, called the URV decomposition, is equally effective in exhibiting the null space and can be updated in O(p/sup 2/) time. The updating technique can be run on a linear array of p processors in O(p) time. >
354 citations
TL;DR: It is shown that nonexponential decay can be parametrized by HSVD (and LPSVD) and under certain conditions the computation time of SVD can be reduced very significantly.
349 citations
TL;DR: This work discusses basic principles of paralled processing, describing the costs of basic operations on parallel machines, including general principles for constructing efficient algorithms, and presents direct and iterative algorithms for solving linear systems of equations, linear least squares problems, the symmetric eigenvalue problem, the nonsymmetric eigene value problem, and the singular value decomposition.
Abstract: We survey general techniques and open problems in numerical linear algebra on parallel architectures. We first discuss basic principles of parallel processing, describing the costs of basic operations on parallel machines, including general principles for constructing efficient algorithms. We illustrate these principles using current architectures and software systems, and by showing how one would implement matrix multiplication. Then, we present direct and iterative algorithms for solving linear systems of equations, linear least squares problems, the symmetric eigenvalue problem, the nonsymmetric eigenvalue problem, the singular value decomposition, and generalizations of these to two matrices. We consider dense, band and sparse matrices.
217 citations
Proceedings Article•
01 Sep 1992
TL;DR: This paper presents a theory and method for decomposing a composite matrix in a meaningful way, proposing that the central issue is rotation extraction, and that the best way to do that is Polar Decomposition.
Abstract: General 3×3 linear or 4×4 homogenous matrices can be formed by composing primitive matrices for translation, rotation, scale, shear, and perspective. Current 3-D computer graphics systems manipulate and interpolate parametric forms of these primitives to generate scenes and motion. For this and other reasons, decomposing a composite matrix in a meaningful way has been a longstanding challenge. This paper presents a theory and method for doing so, proposing that the central issue is rotation extraction, and that the best way to do that is Polar Decomposition. This method also is useful for renormalizing a rotation matrix containing excessive error.
204 citations
TL;DR: A parameterization of the Prony model specific to the radar target identification problem is chosen and several key components to the algorithm, including the use of singular value decomposition and the removal of spurious scattering centers, are presented.
Abstract: A method for characterizing radar signatures using a Prony model is developed based on the concept of scattering centers. A parameterization of the Prony model specific to the radar target identification problem is chosen and several key components to the algorithm, including the use of singular value decomposition and the removal of spurious scattering centers, are presented. The resulting algorithm is tested with data taken from a compact range. These tests include comparison of different targets, different aspect angles and frequency ranges, as well as robustness tests on the algorithm and evaluation of performance in noise. >
193 citations
TL;DR: In this paper, a constructive proof of the existence of the rank-revealing QR factorization of any matrix A of size m x n with numerical rank r is given. But it is not clear how to find a rank revealing RRQR of A if A has numerical rank deficiency.
Abstract: T. Chan has noted that, even when the singular value decomposition of a matrix A is known, it is still not obvious how to find a rank-revealing QR factorization (RRQR) of A if A has numerical rank deficiency. This paper offers a constructive proof of the existence of the RRQR factorization of any matrix A of size m x n with numerical rank r . The bounds derived in this paper that guarantee the existence of RRQR are all of order f i ,in comparison with Chan's 0(2"-') . It has been known for some time that if A is only numerically rank-one deficient, then the column permutation l7 of A that guarantees a small rnn in the QR factorization of A n can be obtained by inspecting the size of the elements of the right singular vector of A corresponding to the smallest singular value of A . To some extent, our paper generalizes this well-known result. We consider the interplay between two important matrix decompositions: the singular value decomposition and the QR factorization of a matrix A . In particular, we are interested in the case when A is singular or nearly singular. It is well known that for any A E R m X n (a real matrix with rn rows and n columns, where without loss of generality we assume rn > n) there are orthogonal matrices U and V such that where C is a diagonal matrix with nonnegative diagonal elements: We assume that a, 2 a2 2 . . 2 on 2 0 . The decomposition (0.1) is the singular value decomposition (SVD) of A , and the ai are the singular values of A . The columns of V are the right singular vectors of A , and the columns of U are the left singular vectors of A . Mathematically, in terms of the singular values, Received December 1, 1990; revised February 8, 199 1. 199 1 Mathematics Subject Classification. Primary 65F30, 15A23, 15A42, 15A15.
TL;DR: This is illustrated by showing how the rank revealing QR factorization can be used to compute solutions to rank deficient least squares problems, to perform subset selection, to compute matrix approximations of given rank, and to solve total least square problems.
Abstract: The rank revealing QR factorization of a rectangular matrix can sometimes be used as a reliable and efficient computational alternative to the singular value decomposition for problems that involve rank determination. This is illustrated by showing how the rank revealing QR factorization can be used to compute solutions to rank deficient least squares problems, to perform subset selection, to compute matrix approximations of given rank, and to solve total least squares problems.
TL;DR: Demmel and Kahan as discussed by the authors showed that the standard bidiagonal QR algorithm can be simplified when the shift is zero and, of greater importance, no subtractions occur, allowing very small singular values to be found with (almost) all the accuracy permitted by the data and at no extra cost.
Abstract: In September 1991 J. W. Demmel and W. M. Kahan were awarded the second SIAM prize in numerical linear algebra for their paper ‘Accurate Singular Values of Bidiagonal Matrices’ [1], referred to as DK hereafter. Among several valuable results was the observation that the standard bidiagonal QR algorithm used in LINPACK [2], and in many other SVD programs, can be simplified when the shift is zero and, of greater importance, no subtractions occur. The last feature permits very small singular values to be found with (almost) all the accuracy permitted by the data and at no extra cost.
TL;DR: This paper extends the well-known QR updating scheme to a similar but more versatile and generally applicable scheme for updating the singular value decomposition (SVD), supplemented with a Jacobi-type SVD procedure, where apparently only a few SVD steps after each QR update suffice in order to restore an acceptable approximation for the SVD.
Abstract: In this paper, the well-known QR updating scheme is extended to a similar but more versatile and generally applicable scheme for updating the singular value decomposition (SVD). This is done by supplementing the QR updating with a Jacobi-type SVD procedure, where apparently only a few SVD steps after each QR update suffice in order to restore an acceptable approximation for the SVD. This then results in a reduced computational cost, comparable to the cost for merely QR updating.The usefulness of such an approximate updating scheme when applied to subspace tracking is examined. It is shown how an $\mathcal{O}(n^2 )$ SVD updating algorithm can restore an acceptable approximation at every stage, with a fairly small tracking error of approximately the time variation in $\mathcal{O}(n)$ time steps.Finally, an error analysis is performed, proving that the algorithm is stable, when supplemented with a Jacobi-type reorthogonalization procedure, which can easily be incorporated into the updating scheme.
TL;DR: In this paper, the authors developed linear systems for 2D traveltime tomography in anisotropic media and analyzed the properties of the linear system for quasi-compressional waves and invert both synthetic and real data.
Abstract: SUMMARY
Cross-borehole seismic data have traditionally been analysed by inverting the arrival times for velocity structure (traveltime tomography). The presence of anisotropy requires that tomographic methods be generalized to account for anisotropy. This generalization allows geological structure to be correctly imaged and allows the anisotropy to be evaluated. In a companion paper we developed linear systems for 2-D traveltime tomography in anisotropic media. In this paper we analyse the properties of the linear system for quasi-compressional waves and invert both synthetic and real data. Solutions to the linear systems consist of estimates of the spatial distributions of five parameters, each corresponding to a linear combination of a small subset of the 21 elastic, anisotropic velocity parameters. The parameters describe the arrival times in the presence of weak anisotropy with arbitrary symmetries. However, these parameters do not, in general, describe the full nature of the anisotropy. The parameters must be further interpreted using additional information on the symmetry system. In the examples in this paper we assume transverse isotropy (TI) in order to interpret our inversions, but it should be noted that this final interpretation could be reformulated in more general terms.
The singular value decomposition of the linear system for traveltime tomography in anisotropic media reveals the (expected) ill-conditioning of these systems. As in isotropic tomography, ill-conditioning arises due to the limited directional coverage that can be achieved when sources and receivers are located in vertical boreholes. In contrast to isotropic tomography, the scalelength of the parametrization controls the nature of the parameter space eigenvectors: with a coarse grid all five parameters are required to model the data; with a fine grid some of the parameters appear only in the null space.
The linear systems must be regularized using external, a priori information. An important regularization is the expectation that the elastic properties vary smoothly (an ad hoc recognition of the insensitivity of the arrival times to the fine-grained properties of the medium). The expectation of smoothness is incorporated by using a regularization matrix that penalizes rough solutions using finite difference penalty terms. The roughness penalty sufficiently constrains the solutions to allow the smooth eigenvectors in the null space of the unconstrained problem to contribute to the solutions. Hence, the spatial distribution of all five parameters is recovered. The level of regularization required is difficult to estimate; we advocate the analysis of a suite of solutions. Plots of the solution roughness against the data residuals can be used to find ‘knee points’, but for the fine tuning of the regularization one has little recourse but to examine a suite of images and use geological plausibility as an additional criterion.
The application of the regularized numerical scheme to the synthetic data reveals that the roughness penalty should include terms that penalize high gradients addition to penalizing high second derivatives. Only when this constraint was included were the features of the original model recovered. The inversions of the field data yield good images of the expected stratigraphy and confirm previous estimates of the magnitude of the anisotropy and the orientation of the symmetry axis. The solutions further indicate an increase in anisotropy from the top to the bottom of the survey region that was not previously detected.
Journal Article•
TL;DR: The rank revealing QR factorization of a rectangular matrix can sometimes be used as a reliable and efficient computational alternative to the singular value decomposition for problems that involve rank determination as discussed by the authors, which can be used to compute solutions to rank deficient least squares problems, to perform subset selection, to compute matrix approximations of given rank, and to solve total least square problems.
Abstract: The rank revealing QR factorization of a rectangular matrix can sometimes be used as a reliable and efficient computational alternative to the singular value decomposition for problems that involve rank determination. This is illustrated by showing how the rank revealing QR factorization can be used to compute solutions to rank deficient least squares problems, to perform subset selection, to compute matrix approximations of given rank, and to solve total least squares problems.
TL;DR: In this article, a signal processing method based on the singular value decomposition (SVD) of a rectangular matrix has been proposed to disentangle the poloidal structure without a priori knowledge.
Abstract: The spatial structure and temporal evolution of coherent MHD modes in fusion plasma devices have been so far inferred from the experimental signals by using spectral techniques. Considering the data as a collection of n-dimensional discrete time series xi(t) permits one to introduce in this context a signal processing method based on the singular value decomposition (SVD) of a rectangular matrix. This paper shows that, whereas the SVD is equivalent to the discrete Fourier transform in the case of travelling sinusoidal waves, in more realistic cases it is a clear improvement on the spectral methods, since it disentangles the poloidal structure without a priori knowledge. Various applications of SVD, first on artificial signals and then on magnetic and SXR signals detected in the JET Tokamak in several plasma regimes, are shown with the purpose of illustrating the power and limits of the method.
01 Sep 1992
TL;DR: In this article, the analysis of canonical correlations of matrix pairs and their numerical computation was studied. But the authors focused on the case of large sparse or structured matrices. And they only considered matrix pairs having the same number of rows which explicitly exhibited the canonical correlations.
Abstract: This paper is concerned with the analysis of canonical correlations of matrix pairs and their numerical computation. We first develop a decomposition theorem for matrix pairs having the same number of rows which explicitly exhibits the canonical correlations. We then present a perturbation analysis of the canonical correlations, which compares favorably with the classical first order perturbation analysis. Then we propose several numerical algorithms for computing the canonical correlations of general matrix pairs; emphasis is placed on the case of large sparse or structured matrices.
TL;DR: In this article, the authors developed an algorithm for adaptively estimating the noise subspace of a data matrix, as is required in signal processing applications employing the signal subspace approach, using a rank-revealing QR factorization instead of the more expensive singular value or eigenvalue decompositions.
Abstract: The authors develop an algorithm for adaptively estimating the noise subspace of a data matrix, as is required in signal processing applications employing the 'signal subspace' approach. The noise subspace is estimated using a rank-revealing QR factorization instead of the more expensive singular value or eigenvalue decompositions. Using incremental condition estimation to monitor the smallest singular values of triangular matrices, the authors can update the rank-revealing triangular factorization inexpensively when new rows are added and old rows are deleted. Experiments demonstrate that the new approach usually requires O(n/sup 2/) work to update an n*n matrix, and that it accurately tracks the noise subspace. >
TL;DR: In this article, the reliability of singular-value decomposition for reconstructing the attractor from one-dimensional chaotic time series has been investigated and doubts about its reliability are confirmed: singular value decomposition can bring distorted and misleading results when nonlinear structures are studied.
TL;DR: It is shown that there can be no simple relationship between distance to singularity and condition number, because the condition number can be computed in pol...
Abstract: The singular value decomposition of a square matrix A answers two questions First, it measures the distance from A to the nearest singular matrix, measuring distance with the two-norm It also computes the condition number, or the sensitivity of $A^{ - 1} $ to perturbations in A, where sensitivity is also measured with the two-norm As is well known, these two quantities, the minimum distance to singularity and the condition number, are essentially reciprocals Using the algorithm of Golub and Kahan [SIAM J Numer Anal, Ser B, 2 (1965), pp 205–224] and its descendants, these quantities may be computed in $O(n^3 )$ operations More recent sensitivity analysis extends this analysis to perturbations of different maximum sizes in each entry of A One may again ask about distance to singularity, condition numbers, and complexity in this new context It is shown that there can be no simple relationship between distance to singularity and condition number, because the condition number can be computed in pol
10 May 1992
TL;DR: A novel two-step, two-sided unitary transformation scheme, tailored to the use of CORDIC (coordinate rotation digital computer) algorithms for high-speed arithmetic, is employed to diagonalize a complex 2*2 matrix.
Abstract: A systolic algorithm for the SVD (singular value decomposition) of arbitrary complex matrices based on the cyclic Jacobi method with parallel ordering is presented. A novel two-step, two-sided unitary transformation scheme, tailored to the use of CORDIC (coordinate rotation digital computer) algorithms for high-speed arithmetic, is employed to diagonalize a complex 2*2 matrix. Architecturally, the complex SVD array is modeled on the Brent-Luk-VanLoan array for real SVD. An expandable array of O(n/sup 2/) complex 2*2 matrix processors computes the SVD of an n*n matrix in O(n log n) time. A CORDIC architecture for the complex 2*2 processor with an area complexity twice that of a real 2*2 processor is proposed. Computation time for the complex SVD array is less than three times that for a real SVD array with a similar CORDIC-based implementation. >
TL;DR: A general technique for solving homogeneous matrix equations as applied to numerical modeling procedures in microwave and millimeter-wave structures is introduced in this article, where the pole-free determinant function is eliminated.
Abstract: A general technique for solving homogeneous matrix equations as applied to numerical modeling procedures in microwave and millimeter-wave structures is introduced. By using singular value decomposition, well-known numerical problems related to poles and steep gradients in the determinant function are eliminated. The proposed technique is generally applicable, improves the accuracy and reliability of computed results, and significantly reduces the CPU time due to a more moderate behavior of the function to be analyzed. A dispersion characteristics example of a conductor-backed slotline MMIC structure illustrates the advantage of the pole-free formulation over conventional determinant calculations. >
TL;DR: Tradeoffs between resolution and error in estimating source voxel intensities are discussed, and estimates of these errors provide a robust means of stabilizing the solution to the ill-posed inverse problem.
Abstract: Generalized matrix inverses are used to estimate source activity distributions from single photon emission computed tomography (SPECT) projection measurements. Image reconstructions for a numerical simulation and a clinical brain study are examined. The photon flux from the source region and photon detection by the gamma camera are modeled by matrices which are computed by Monte Carlo methods. The singular value decompositions (SVDs) of these matrices give considerable insight into the SPECT image reconstruction problem and the SVDs are used to form approximate generalized matrix inverses. Tradeoffs between resolution and error in estimating source voxel intensities are discussed, and estimates of these errors provide a robust means of stabilizing the solution to the ill-posed inverse problem. In addition to its quantitative clinical applications, the generalized matrix inverse method may be a useful research tool for tasks such as evaluating collimator design and optimizing gamma camera motion. >
01 Jan 1992
TL;DR: Two new subspace algorithms for identifying mixed deterministic-stochastic systems are derived and these state sequences are shown to be outputs of nonsteady-state Kalman filter banks.
Abstract: Two new subspace algorithms for identifying mixed deterministic-stochastic systems are derived. Both algorithms determine state sequences through the projection of input and output data. These state sequences are shown to be outputs of nonsteady-state Kalman filter banks. From these it is easy to determine the state space system matrices. The algorithms are always convergent (noninterative) and numerically stable since they only make use of QR and singular value decompositions. The two algorithms are similar, but the second one trades off accuracy for simplicity. An example involving a glass oven is considered. >
01 Jan 1992
TL;DR: This paper discusses multimatrix generalizations of two well-known orthogonal rank factorizations of a matrix: the generalized singular value decomposition and the generalized QR-(or URV-) decomposition.
Abstract: This paper discusses multimatrix generalizations of two well-known orthogonal rank factorizations of a matrix: the generalized singular value decomposition and the generalized QR-(or URV-) decomposition. These generalizations can be obtained for any number of matrices of compatible dimensions. This paper discusses in detail the structure of these generalizations and their mutual relations and gives a constructive proof for the generalized QR-decompositions.
TL;DR: In this article, a smooth singular value decomposition for a matrix which is smoothly dependent on a parameter is presented. But this decomposition is based on a system of ordinary differential equations and it is shown that the numerical solution of an initial value problem associated with these differential equations provides a feasible approach to the solution of this problem.
Abstract: This paper is concerned with finding a smooth singular value decomposition for a matrix which is smoothly dependent on a parameter. A previous approach to this problem was based on minimisation techniques, here, in contrast, a system of ordinary differential equations is derived for the decomposition. It is shown that the numerical solution of an initial value problem associated with these differential equations provides a feasible approach to the solution of this problem. Particular consideration is given to the situation which arises with equal modulus singular values which lead to indeterminacies in the evaluations needed for the numerical solution. Examples which illustrate the behaviour of the method are included.
TL;DR: For discrete systems, the set of all state covariances X which can be assigned to the closed-loop system via a dynamic controller is characterized explicitly in this article, where the state covariance X is parameterized with an arbitrary orthonormal matrix U of proper dimension.
Abstract: For discrete systems, the set of all state covariances X which can be assigned to the closed-loop system via a dynamic controller is characterized explicitly. For any assignable state covariance X, the set of all controllers that assign this X to the closed-loop system is parameterized with an arbitrary orthonormal matrix U of proper dimension. >
TL;DR: In this article, an analysis of the solutions of the total least squares problem (TLS) in cases where the matrix A,B may have multiple smallest singular values is presented, and the difference between the TLS and the LS solutions is obtained; the error bounds for the perturbed TLS solutions with or without minimal length are deduced.
Abstract: This paper presents an analysis of the solutions of the total least squares problem (TLS) $AX \approx B$ in cases where the matrix $(A,B)$ may have multiple smallest singular values. General formulas for the minimum norm TLS solutions are given; the difference between the TLS and the LS solutions is obtained; the error bounds for the perturbed TLS solutions with or without minimal length are deduced. The analysis is useful especially for rank deficient problems and generalizes previous results of Golub and Van Loan, Van Huffel and Vandewalle, and Zoltowski. Numerical results for a practical application are also given to verify the error bounds.