Showing papers on "Singular value decomposition published in 1994"

Optimization and Dynamical Systems

Abstract: Contents: Matrix Eigenvalue Methods.- Double Bracket Isospectral Flows.- Singular Value Decomposition.- Linear Programming.- Approximation and Control.- Balanced Matrix Factorizations.- Invariant Theory and System Balancing.- Balancing via Gradient Flows.- Sensitivity Optimization.- Linear Algebra.- Dynamical Systems.- Global Analysis.

A paraperspective factorization method for shape and motion recovery

Abstract: The factorization method, first developed by Tomasi and Kanade, recovers both the shape of an object and its motion from a sequence of images, using many images and tracking many feature points to obtain highly redundant feature position information. The method robustly processes the feature trajectory information using singular value decomposition (SVD), taking advantage of the linear algebraic properties of orthographic projection. However, an orthographic formulation limits the range of motions the method can accommodate. Paraperspective projection, first introduced by Ohta, is a projection model that closely approximates perspective projection by modelling several effects not modelled under orthographic projection, while retaining linear algebraic properties. We have developed a paraperspective factorization method that can be applied to a much wider range of motion scenarios, such as image sequences containing significant translational motion toward the camera or across the image. We present the results of several experiments which illustrate the method's performance in a wide range of situations, including an aerial image sequence of terrain taken from a low-altitude airplane.

Analysis of 3-D rotation fitting

Abstract: Computational techniques for fitting a 3-D rotation to 3-D data are recapitulated in a refined form as minimization over proper rotations, extending three existing methods-the method of singular value decomposition, the method of polar decomposition, and the method of quaternion representation. Then, we describe the problem of 3-D motion estimation in this new light. Finally, we define the covariance matrix of a rotation and analyze the statistical behavior of errors in 3-D rotation fitting. >

On singular Wishart and singular multivariate beta distributions

Abstract: This paper extends the study of Wishart and multivariate beta distributions to the singular case, where the rank is below the dimensionality The usual conjugacy is extended to this case A volume element on the space of positive semidefinite $m \times m$ matrices of rank $n < m$ is introduced and some transformation properties established The density function is found for all rank-$n$ Wishart distributions as well as the rank-1 multivariate beta distribution To do that, the Jacobian for the transformation to the singular value decomposition of general $m \times n$ matrices is calculated The results in this paper are useful in particular for updating a Bayesian posterior when tracking a time-varying variance-covariance matrix

Total least squares for affinely structured matrices and the noisy realization problem

Abstract: Structured rank-deficient matrices arise in many applications in signal processing, system identification, and control theory. The author discusses the structured total least squares (STLS) problem, which is the problem of approximating affinely structured matrices (i.e., matrices affine in the parameters) by similarly structured rank-deficient ones, while minimizing an L/sub 2/-error criterion. It is shown that the optimality conditions lead to a nonlinear generalized singular value decomposition, which can be solved via an algorithm that is inspired by inverse iteration. Next the author concentrates on the so-called L/sub 2/-optimal noisy realization problem, which is equivalent with approximating a given data sequence by the impulse response of a finite dimensional, time invariant linear system of a given order. This can be solved as a structured total least squares problem. It is shown with some simple counter examples that "classical" algorithms such as the Steiglitz-McBride (1965), iterative quadratic maximum likelihood and Cadzow's (1988) iteration do not converge to the optimal L/sub 2/ solution, despite misleading claims in the literature. >

Singular Value Decomposition

Abstract: Many numerical methods used in application areas such as signal processing, estimation, and control are based on the singular value decomposition (SVD) of matrices. The SVD is widely used in least squares estimation, systems approximations, and numerical linear algebra.

The biorthogonal decomposition as a tool for investigating fluctuations in plasmas

Abstract: The investigation of fluctuation phenomena in plasmas often necessitates the analysis of spatiotemporal signals. It is shown how such signals can be analyzed using the biorthogonal decomposition, which splits them into orthogonal spatial and temporal modes. The method, also referred to as the singular value decomposition, allows complex spatiotemporal patterns to be decomposed into a few coherent modes that are often easier to interpret. This is illustrated with two applications to fluctuating soft x‐ray and magnetic signals, as measured in a tokamak. Emphasis is given to the physical interpretation of the biorthogonal components and their link with known physical models is discussed. It is shown how new insight can be gained in the interpretation of spatiotemporal plasma dynamics.

Dynamically adaptive MRI with encoding by singular value decomposition

Abstract: A new MRI spatial encoding method based upon the singular value decomposition (SVD) and using spatially selective RF excitation is described. This encoding technique is particularly applicable to dynamic adaptive MRI, because it provides a near minimal set of spatial encoding profiles computed using an image estimate that is determined from a previously obtained image. Experimental results are presented for two cases, which exemplify its potential use in different dynamic imaging tasks. SVD-encoded MRI has demonstrated to be a highly efficient encoding scheme.

K-means clustering in a low-dimensional Euclidean space

Abstract: A procedure is developed for clustering objects in a low-dimensional subspace of the column space of an objects by variables data matrix. The method is based on the K-means criterion and seeks the subspace that is maximally informative about the clustering structure in the data. In this low-dimensional representation, the objects, the variables and the cluster centroids are displayed jointly. The advantages of the new method are discussed, an efficient alternating least-squares algorithm is described, and the procedure is illustrated on some artificial data.

A Frobenius norm approach to glottal closure detection from the speech signal

Abstract: The detection of glottal closure instants has been a necessary step in several applications of speech processing, such as voice source analysis, speech prosody manipulation and speech synthesis. The paper presents a new algorithm for glottal closure detection that compares favorably with other methods available in terms of robustness and computational efficiency. The authors propose to use the singular value decomposition (SVD) approach to detect the instants of glottal closure from the speech signal. The proposed SVD method amounts to calculating the Frobenius norms of signal matrices and therefore is computationally efficient. Moreover, it produces well-defined and reliable peaks that indicate the instants of glottal closure. Finally, with the introduction of the total linear least squares technique, two other proposed methods are reinvestigated and unified into the SVD framework. >

Efficient processing of nmr echo trains

Abstract: A method and apparatus (28) is disclosed for efficient processing of NMR echo trains (fig.3) in well logging. A priori information about the nature of the expected signals is used to obtain an approximation of the signal using a set of pre-selected basis functions (fig.5). A singular decomposition (SVD) is applied to a matrix incorporating information about the basis functions and is stored off-line a memory. During the actual measurement, the apparatus estimates a parameter related to the SNR of the received NMR echo trains and uses it to determine a signal approximation model in conjunction with the SVD of the basis function matrix. This approximation is used to determine in real time attributes (fig. 8) of the earth formation being investigated.

An implicit shift bidiagonalization algorithm for ill-posed problems

Abstract: Iterative methods based on Lanczos bidiagonalization with full reorthogonalization (LBDR) are considered for solving large-scale discrete ill-posed linear least-squares problems of the form minxllAx - bllz. Methods for regularization in the Krylov subspaces are discussed which use generalized cross validation (GCV) for determining the regularization parameter. These methods have the advantage that no a priori information about the noise level is required. To improve convergence of the Lanczos process we apply a variant of the implicitly restarted Lanczos algorithm by Sorensen using zero shifts. Although this restarted method simply corresponds to using LBDR with a starting vector (AAr~'b, it is shown that carrying out the process implicitly is essential for numerical stability. An LBDR algorithm is presented which incorporates implicit restarts to ensure that the global minimum of the CGV curve corresponds to a minimum on the curve for the truncated SVD solution. Numerical results are given comparing the performance of this algorithm with nonrestarted LBDR.

Large Least Squares Problems Involving Kronecker Products

Abstract: The general problem considered here is the least squares solution of $(A \otimes B)x = t$, where $A$ and $B$ are full rank, rectangular matrices, and $A \otimes B$ is the Kronecker product of $A$ and $B$. Equations of this form arise in areas such as digital image and signal processing, photogrammetry, finite elements, and multidimensional approximation. An efficient method of solution is based on QR factorizations of the original matrices $A$ and $B$. It is demonstrated how these factorizations can be used to obtain the Cholesky factorization of the least squares coefficient matrix without explicitly forming the normal equations. A similar approach based on singular value decomposition (SVD) factorizations also is indicated for the rank-deficient case.

Spectral decomposition of time-frequency distribution kernels

Abstract: This paper addresses the general problem of approximating a given time-frequency distribution (TFD) in terms of other distributions with desired properties It relates the approximation of two time-frequency distributions to their corresponding kernel approximation It is shown that the singular-value decomposition (SVD) of the time-frequency (t-f) kernels allows the expression of the time-frequency distributions in terms of weighted sum of smoothed pseudo Wigner-Ville distributions or modified periodograms, which are the two basic nonparametric power distributions for stationary and nonstationary signals, respectively The windows appearing in the decomposition take zero and/or negative values and, therefore, are different than the time and lag windows commonly employed by these two distributions The centrosymmetry and the time-support properties of the kernels along with the fast decay of the singular values lead to computational savings and allow for an efficient reduced rank kernel approximations >

Iterative SVD-based methods for ill-posed problems

Abstract: Very large matrices with rapidly decaying singular values commonly arise in the numerical solution of ill-posed problems. The singular value decomposition (SVD) is a basic tool for both the analysis and computation of solutions to such problems. In most applications, it suffices to obtain a partial SVD consisting of only the largest singular values and their corresponding singular vectors. In this paper, two separate approaches—one based on subspace iteration and the other based on the Lanczos method—are considered for the efficient iterative computation of partial SVDs. In the context of ill-posed problems, an analytical and numerical comparison of these two methods is made and the role of the regularization operator in convergence acceleration is explored.

A Parallel Algorithm for Computing the Singular Value Decomposition of a Matrix

Abstract: A parallel algorithm for computing the singular value decomposition of a matrix is presented. The algorithm uses a divide and conquer procedure based on a rank one modification of a bidiagonal matrix. Numerical difficulties associated with forming the product of a matrix with its transpose are avoided, and numerically stable formulae for obtaining the left singular vectors after computing updated right singular vectors are derived. A deflation technique is described that, together with a robust root finding method, assures computation of the singular values to full accuracy in the residual and also assures orthogonality of the singular vectors.

Comments on the PLS kernel algorithm

Abstract: Lindgren et al. (J. Chemometrics, 7, 45–49 (1993)) published a so-called kernel algorithm for PLS regression of Y against X when the number of objects is very large. The algorithm is based solely on deflation of the cross-product matrices XTX, YTY and XTY. The algorithm is now described in a shorter and more transparent way and compared with a similar algorithm for the singular value decomposition of XTY.

Cross-correlation neural network models

Abstract: In this paper we provide theoretical foundations for a new neural model for singular value decomposition based on an extension of the Hebbian learning rule called the cross-coupled Hebbian rule. The model is extracting the SVD of the cross-correlation matrix of two stochastic signals and is an extension on previous work on neural-network-related principal component analysis (PCA). We prove the asymptotic convergence of the network to the principal (normalized) singular vectors of the cross-correlation and we provide simulation results which suggest that the convergence is exponential. The new model may have useful applications in the problems of filtering for signal processing and signal detection. >

SVD technique for estimation of harmonic components in a power system: a statistical approach

Abstract: The paper presents a statistical approach to the estimation of harmonic components in a power system. Mathematically it is based on singular value decomposition (SVD). Three different techniques are investigated: the standard averaged SVD, the total LS and double SVD. The results of numerical experiments illustrating the features of all these approaches are presented and discussed.

A TQR-iteration based adaptive SVD for real time angle and frequency tracking

Abstract: The transposed VR (TQR) iteration is a square root version of the symmetric QR iteration. The TQR algorithm converges directly to the singular value decomposition (SVD) of a matrix and was originally derived to provide a means to identify and reduce the effects of outliers for robust SVD computation. The paper extends the TQR algorithm to incorporate complex data and weighted norms, formulates a TQR-iteration based adaptive SVD algorithm, develops a real time systolic architecture, and analyzes performance. The applications of high resolution angle and frequency tracking are developed and the updating scheme is so tailored. A deflation mechanism reduces both the computational complexity of the algorithm and the hardware complexity of the systolic architecture, making the method ideal for real time applications. Simulation results demonstrate the performance of the method and compare it to existing SVD tracking schemes. The results show that the method is exceptional in terms of performance to cost ratio and systolic implementation. >

URV ESPRIT for tracking time-varying signals

Abstract: ESPRIT is an algorithm for determining the fixed directions of arrival of a set of narrowband signals at an array of sensors. Unfortunately, its computational burden makes it unsuitable for real time processing of signals with time-varying directions of arrival. The authors develop a new implementation of ESPRIT that has potential for real time processing. It is based on a rank-revealing URV decomposition, rather than the eigendecomposition or singular value decomposition used in previous ESPRIT algorithms. The authors demonstrate its performance on simulated data representing both constant and time-varying signals. They find that the URV-based ESPRIT algorithm is effective for estimating time-varying directions-of-arrival at considerable computational savings over the SVD-based algorithm. >

Balanced realization and model reduction of singular systems

Abstract: The problem of balanced realization and model reduction of a singular system of the form E[xdot] = Ax + Bu, where E is a singular matrix, is considered. Using coordinate transformation, which can be computed by performing singular value decomposition of E, we derive our first approach to the balancing of singular systems. The second approach is based on standard form decomposition of singular systems to slow and fast subsystems and performing balanced realization on the decomposed model. In this sense model reduction can be established in two steps: first, by decomposing the singular system and second, by performing balancing transformation on the decomposed subsystems.

Frequency domain structural system identification by observability range space extraction

Abstract: This paper presents the frequency domain observability range space extraction (FORSE) identification algorithm. FORSE is a singular value decomposition based identification algorithm which constructs a state space model directly from frequency domain data. It is numerically well behaved when applied to multivariable and high dimensional structural systems. It can achieve high modeling accuracy by properly overparametering the model. Its effectiveness for structural systems is demonstrated using the MIT Middeck Active Control Experiment (MACE). MACE is an active structural control experiment to be conducted in the Space Shuttle middeck.

SVD-NET: an algorithm that automatically selects network structure

Abstract: An algorithm is developed for training feedforward neural networks that uses singular value decomposition (SVD) to identify and eliminate redundant hidden nodes. Minimizing redundancy gives smaller networks, producing models that generalize better and thus eliminate the need of using cross-validation to avoid overfitting. The method is demonstrated by modeling a chemical reactor. >

Numerical Gradient Algorithms for Eigenvalue and Singular Value Calculations

Abstract: Recent work has shown that the algebraic question of determining the eigenvalues, or singular values, of a matrix can be answered by solving certain continuous-time gradient flows on matrix manifolds. To obtain computational methods based on this theory, it is reasonable to develop algorithms that iteratively approximate the continuous-time flows. In this paper the authors propose two algorithms, based on a double Lie-bracket equation recently studied by Brockett, that appear to be suitable for implementation in parallel processing environments. The algorithms presented achieve, respectively, the eigenvalue decomposition of a symmetric matrix and the singular value decomposition of an arbitrary matrix. The algorithms have the same equilibria as the continuous-time flows on which they are based and inherit the exponential convergence of the continuous-time solutions.

Fast algorithms for updating signal subspaces

Abstract: In various real-time signal processing and communication applications, it is often required to track a low-dimensional signal subspace that slowly varies with time. Conventional methods of updating the signal subspace rely on eigendecomposition or singular value decomposition, which is computationally expensive and difficult to implement in parallel. Recently, Xu and Kailath proposed fast and parallelizable Lanczos-based algorithms for estimating the signal subspace based on the data matrices or the covariance matrices. In this paper, we shall extend these algorithms to achieve fast tracking of the signal subspace. The computational complexity of the new methods is O(M/sup 2/d) per update, where M is the size of the data vectors and d is the dimension of the signal subspace. Unlike most tracking methods that assume d is fixed and/or known a priori, the new methods also update the signal subspace dimension. More importantly, under certain stationarity conditions, we can show that the Lanczos-based methods are asymptotically equivalent to the more costly SVD or eigendecomposition based methods and that the estimation of d is strongly consistent. Knowledge of the previous signal subspace estimate is incorporated to achieve better numerical properties for the current signal subspace estimate. Numerical simulations for some signal scenarios are also presented. >