Showing papers on "Matrix decomposition published in 2003"

Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later ⁄

Abstract: In principle, the exponential of a matrix could be computed in many ways. Methods involving approximation theory, differential equations, the matrix eigenvalues, and the matrix characteristic polyn...

Document clustering based on non-negative matrix factorization

Abstract: In this paper, we propose a novel document clustering method based on the non-negative factorization of the term-document matrix of the given document corpus. In the latent semantic space derived by the non-negative matrix factorization (NMF), each axis captures the base topic of a particular document cluster, and each document is represented as an additive combination of the base topics. The cluster membership of each document can be easily determined by finding the base topic (the axis) with which the document has the largest projection value. Our experimental evaluations show that the proposed document clustering method surpasses the latent semantic indexing and the spectral clustering methods not only in the easy and reliable derivation of document clustering results, but also in document clustering accuracies.

A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization

Abstract: In this paper, we present a nonlinear programming algorithm for solving semidefinite programs (SDPs) in standard form. The algorithm's distinguishing feature is a change of variables that replaces the symmetric, positive semidefinite variable X of the SDP with a rectangular variable R according to the factorization X=RR T . The rank of the factorization, i.e., the number of columns of R, is chosen minimally so as to enhance computational speed while maintaining equivalence with the SDP. Fundamental results concerning the convergence of the algorithm are derived, and encouraging computational results on some large-scale test problems are also presented.

Non-negative matrix factorization for polyphonic music transcription

Abstract: We present a methodology for analyzing polyphonic musical passages comprised of notes that exhibit a harmonically fixed spectral profile (such as piano notes). Taking advantage of this unique note structure, we can model the audio content of the musical passage by a linear basis transform and use non-negative matrix decomposition methods to estimate the spectral profile and the temporal information of every note. This approach results in a very simple and compact system that is not knowledge-based, but rather learns notes by observation.

When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts?

Abstract: We interpret non-negative matrix factorization geometrically, as the problem of finding a simplicial cone which contains a cloud of data points and which is contained in the positive orthant. We show that under certain conditions, basically requiring that some of the data are spread across the faces of the positive orthant, there is a unique such simplicial cone. We give examples of synthetic image articulation databases which obey these conditions; these require separated support and factorial sampling. For such databases there is a generative model in terms of 'parts' and NMF correctly identifies the 'parts'. We show that our theoretical results are predictive of the performance of published NMF code, by running the published algorithms on one of our synthetic image articulation databases.

A direct approach for distribution system load flow solutions

Abstract: A direct approach for unbalanced three-phase distribution load flow solutions is proposed in this paper. The special topological characteristics of distribution networks have been fully utilized to make the direct solution possible. Two developed matrices-the bus-injection to branch-current matrix and the branch-current to bus-voltage matrix-and a simple matrix multiplication are used to obtain load flow solutions. Due to the distinctive solution techniques of the proposed method, the time-consuming LU decomposition and forward/backward substitution of the Jacobian matrix or Y admittance matrix required in the traditional load flow methods are no longer necessary. Therefore, the proposed method is robust and time-efficient. Test results demonstrate the validity of the proposed method. The proposed method shows great potential to be used in distribution automation applications.

MMSE extension of V-BLAST based on sorted QR decomposition

Abstract: In rich-scattering environments, layered space-time architectures like the BLAST system may exploit the capacity advantage of multiple antenna systems. We present a novel, computationally efficient algorithm for detecting V-BLAST architectures with respect to the MMSE criterion. It utilizes a sorted QR decomposition of the channel matrix and leads to a simple successive detection structure. The new algorithm needs only a fraction of computational effort compared to the standard V-BLAST algorithm and achieves the same error performance.

On cones of nonnegative quadratic functions

Abstract: We derive linear matrix inequality (LMI) characterizations and dual decomposition algorithms for certain matrix cones which are generated by a given set using generalized co-positivity. These matrix cones are in fact cones of nonconvex quadratic functions that are nonnegative on a certain domain. As a domain, we consider for instance the intersection of a (upper) level-set of a quadratic function and a half-plane. Consequently, we arrive at a generalization of Yakubovich's S-procedure result. Although the primary concern of this paper is to characterize the matrix cones by LMIs, we show, as an application of our results, that optimizing a general quadratic function over the intersection of an ellipsoid and a half-plane can be formulated as semidefinite programming (SDP), thus proving the polynomiality of this class of optimization problems, which arise, e.g., from the application of the trust region method for nonlinear programming. Other applications are in control theory and robust optimization.

An algebraic geometric approach to the identification of a class of linear hybrid systems

Abstract: We propose an algebraic geometric solution to the identification of a class of linear hybrid systems. We show that the identification of the model parameters can be decoupled from the inference of the hybrid state and the switching mechanism generating the transitions, hence we do not constraint the switches to be separated by a minimum dwell time. The decoupling is obtained from the so-called hybrid decoupling constraint, which establishes a connection between linear hybrid system identification, polynomial factorization and hyperplane clustering. In essence, we represent the number of discrete states n as the degree of a homogeneous polynomial p and the model parameters as factors of p. We then show that one can estimate n from a rank constraint on the data, the coefficients of p from a linear system, and the model parameters from the derivatives of p. The solution is closed form if and only if n/spl les/4. Once the model parameters have been identified, the estimation of the hybrid state becomes a simpler problem. Although our algorithm is designed for noiseless data, we also present simulation results with noisy data.

Linear Inverse Problems in Structural Econometrics Estimation based on spectral decomposition

Abstract: Inverse problems can be described as functional equations where the value of the function is known or easily estimable but the argument is unknown. Many problems in econometrics can be stated in the form of inverse problems where the argument itself is a function. For example, consider a nonlinear regression where the functional form is the object of interest. One can readily estimate the conditional expectation of the dependent variable given a vector of instruments. From this estimate, one would like to recover the unknown functional form. This chapter provides an introduction to the estimation of the solution to inverse problems. It focuses mainly on integral equations of the first kind. Solving these equations is particularly challenging as the solution does not necessarily exist, may not be unique, and is not continuous. As a result, a regularized (or smoothed) solution needs to be implemented. We review different regularization methods and study the properties of the estimator. Integral equations of the first kind appear, for example, in the generalized method of moments when the number of moment conditions is infinite, and in the nonparametric estimation of instrumental variable regressions. In the last section of this chapter, we investigate integral equations of the second kind, whose solutions may not be unique but are continuous. Such equations arise when additive models and measurement error models are estimated nonparametrically.

Reduced complexity MMSE detection for BLAST architectures

Abstract: Theoretical and experimental studies have shown that layered space-time architectures like the BLAST system can exploit the capacity advantage of multiple antenna systems in rich-scattering environments. We present a new efficient algorithm for detecting such architectures with respect to the MMSE criterion. This algorithm utilizes a sorted QR decomposition of the channel matrix and leads to a simple successive detection structure. The algorithm needs only a fraction of the computational effort compared to the standard V-BLAST algorithm and achieves the same bit error performance.

Loop star basis functions and a robust preconditioner for EFIE scattering problems

Abstract: An electric field integral equation (EFIE) formulation using the loop-star basis functions has been developed for modeling plane wave scattering from perfect conducting objects. A stability analysis at the DC limit shows that the use of the Rao-Wilton-Glisson (RWG) basis functions results in a singular matrix operator. However, the use of the loop-star basis functions results in a well-conditioned matrix. Moreover, a preconditioner constructed from a two-step process, based on near interactions and an incomplete factorization with a heuristic drop strategy, has been proposed in conjunction with the conjugate gradient method to solve the resulting matrix equation. The approach is shown to be effective for resolving both the low frequency instability and the bad conditioning of the EFIE method. The computational complexity of the proposed approach is shown to be O(N/sup 2/).

Multidimensional Systems Theory and Applications

Abstract: List of Acronyms List of Notations Preface Acknowledgements Introduction 1: Trends in Multidimensional Systems Theory 1 Introduction 2 Multidimensional Systems Stability 3 Multivariate Realization Theory 4 n-D Problem of Moments and its Applications in Multidimensional Systems Theory 5 Role of Irreducible Polynomials in Multidimensional Systems Theory 6 Hilbert Transform and Spectral Factorization 7 Conclusions 8 Updates 2: Causal and Weakly Causal 2-D Filters with Applications in Stabilization 1 Scalar 2-D Input / output Systems 2 Stability 3 Structural Stability 4 Multi-Input Multi-Output Systems 5 Stabilization of Scalar Systems 6 Characterization of Stabilizers for Scalar Systems 7 Stabilization of Strictly Causal Transfer Matrices 8 Characterization of Stabilizers for MIMO Systems 9 Stabilization of Weakly Causal Systems 10 Stabilization of MIMO Weakly Causal Systems 11 Conclusions 12 Updates 3: The Equation Ax = b over the Ring C[z, w] 1 Introduction 2 Sufficient Condition for Solution 3 Appendix A Zero-Dimensional Polynomial Ideals 4: Groebner Bases: An Algorithmic Method in Polynomial Ideal Theory 1 Introduction 2 Groebner Bases 3 Algorithmic Construction of Groebner Bases 4 An Improved Version of the Algorithm 5 Application: Canonical Simplification, Decision of Ideal Congruence and Membership, Computation inResidue Class Rings 6 Application: Solvability and Exact Solution of Systems of Algebraic Equations 7 Application: Solution of Linear Homogeneous Equations with Polynomial Coefficients 8 Groebner Bases for Polynomial Ideals over the Integers 9 Other Applications 10 Specializations, Generalizations, Implementations, Complexity 11 Updates 5: Multivariate Polynomials, Matrices, and Matrix-Fraction Descriptions 1 Introduction 2 Relative Primeness and GCD Extraction from Multivariate Polynomials 3 Polynomial Matrix Primitive Factorization in the Bivariate Case 4 Multivariate Polynomial Matrix Factorization 5 Computations for Coprimeness Using Groebner Bases 6 Generalization of the Serre Conjecture and its Consequences 7 Factorization as a Product of Elementary Matrix Factors 8 Applications in Multidimensional Systems Stabilization 9 Behavioral Approach 10 Conclusions 6: Recent Impacts of Multidimensional Systems Research 1 Introduction 2 Inference of Stability of Sets of Multidimensional Systems from Subsets of Low Cardinality 3 Multiple Deconvolution Operators for Robust Superresolution 4 Multisensor Array-Based Superresolution 5 Wavelets for Superresolution 6 Other Recent Applications 7 Conclusions 7: Multivariate Rational Approximants of the Pade Type 1 Introduction and Motivation 2 Multivariate Pade-Type Approximants (Scalar Case) 3 Pade Type Matrix Approximants 4 Conclusions

Summarizing popular music via structural similarity analysis

Abstract: We present a framework for summarizing digital media based on structural analysis. Though these methods are applicable to general media, we concentrate here on characterizing the repetitive structure in popular music. In the first step, a similarity matrix is calculated from interframe spectral similarity. Segment boundaries, such as verse-chorus transitions, are found by correlating a kernel along the diagonal of the matrix. Once segmented, spectral statistics of each segment are computed. In the second step, segments are clustered, based on the pairwise similarity of their statistics, using a matrix decomposition. Finally, the audio is summarized by combining segments representing the clusters most frequently repeated throughout the piece. We present results on a small corpus showing more than 90% correct detection of verse and chorus segments.

Unsupervised Color Decomposition Of Histologically Stained Tissue Samples

Abstract: Accurate spectral decomposition is essential for the analysis and diagnosis of histologically stained tissue sections. In this paper we present the first automated system for performing this decomposition. We compare the performance of our system with ground truth data and report favorable results.

Non-negative matrix factorization for visual coding

Abstract: This paper combines linear spun coding and nonnegative matrix factorization into sparse non-negative matrix factorization. In contrast to non-negative matrix factorization, the new model can learn much sparser representation via imposing sparseness constraints explicitly; in contrast to a close model -non-negative sparse coding, the new model can learn parts-based representation via fully multiplicative updates because of adapting a generalized Kullback-Leibler divergence instead of the conventional mean error for approximation error. Experiments on MIT-CBCL training facts data demonstrate the effectiveness of the proposed method.

Spectral AMGe ($\rho$AMGe)

Abstract: We introduce spectral element-based algebraic multigrid ($\rho$AMGe), a new algebraic multigrid method for solving systems of algebraic equations that arise in Ritz-type finite element discretizations of partial differential equations. The method requires access to the element stiffness matrices, which enables accurate approximation of algebraically "smooth" vectors (i.e., error components that relaxation cannot effectively eliminate). Most other algebraic multigrid methods are based in some manner on predefined concepts of smoothness. Coarse-grid selection and prolongation, for example, are often defined assuming that smooth errors vary slowly in the direction of "strong" connections (relatively large coefficients in the operator matrix). One aim of $\rho$AMGe is to broaden the range of problems to which the method can be successfully applied by avoiding any implicit premise about the nature of the smooth error. $\rho$AMGe uses the spectral decomposition of small collections of element stiffness matrices to determine local representations of algebraically smooth error components. This provides a foundation for generating the coarse level and for defining effective interpolation. This paper presents a theoretical foundation for $\rho$AMGe along with numerical experiments demonstrating its robustness.

J-Orthogonal Matrices: Properties and Generation

Abstract: A real, square matrix Q is J-orthogonal if QTJQ = J, where the signature matrix $J = \diag(\pm 1)$. J-orthogonal matrices arise in the analysis and numerical solution of various matrix problems involving indefinite inner products, including, in particular, the downdating of Cholesky factorizations. We present techniques and tools useful in the analysis, application, and construction of these matrices, giving a self-contained treatment that provides new insights. First, we define and explore the properties of the exchange operator, which maps J-orthogonal matrices to orthogonal matrices and vice versa. Then we show how the exchange operator can be used to obtain a hyperbolic CS decomposition of a J-orthogonal matrix directly from the usual CS decomposition of an orthogonal matrix. We employ the decomposition to derive an algorithm for constructing random J-orthogonal matrices with specified norm and condition number. We also give a short proof of the fact that J-orthogonal matrices are optimally scaled und...

Adaptive overrelaxed bound optimization methods

Abstract: We study a class of overrelaxed bound optimization algorithms, and their relationship to standard bound optimizers, such as Expectation-Maximization, Iterative Scaling, CCCP and Non-Negative Matrix Factorization. We provide a theoretical analysis of the convergence properties of these optimizers and identify analytic conditions under which they are expected to outperform the standard versions. Based on this analysis, we propose a novel, simple adaptive overrelaxed scheme for practical optimization and report empirical results on several synthetic and real-world data sets showing that these new adaptive methods exhibit superior performance (in certain cases by several times speedup) compared to their traditional counterparts. Our extensions are simple to implement, apply to a wide variety of algorithms, almost always give a substantial speedup, and do not require any theoretical analysis of the underlying algorithm.

Analysis of Nonsmooth Symmetric-Matrix-Valued Functions with Applications to Semidefinite Complementarity Problems

Abstract: For any function f from $\mathbb R$ to $\mathbb R$, one can define a corresponding function on the space of n × n (block-diagonal) real symmetric matrices by applying f to the eigenvalues of the spectral decomposition. We show that this matrix-valued function inherits from f the properties of continuity, (local) Lipschitz continuity, directional differentiability, Frechet differentiability, continuous differentiability, as well as ($\rho$-order) semismoothness. Our analysis uses results from nonsmooth analysis as well as perturbation theory for the spectral decomposition of symmetric matrices. We also apply our results to the semidefinite complementarity problem, addressing some basic issues in the analysis of smoothing/semismooth Newton methods for solving this problem.

Sparse component analysis for blind source separation with less sensors than sources

Abstract: A sparse decomposition approach of observed data matrix is presented in this paper and the approach is then used in blind source separation with less sensors than sources. First, sparse representation (factorization) of a data matrix is discussed. For a given basis matrix, there exist infinite coefficient matrices (solutions) generally such that the data matrix can be represented by the product of the basis matrix and coefficient matrices. However, the sparse solution with minimum1-norm is unique with probability one, and can be obtained by using linear programming algorithm. The basis matrix can be estimated using gradient type algorithm or Kmeans clustering algorithm. Next, blind source separation is discussed based on sparse factorization approach. The blind separation technique includes two steps, one is to estimate a mixing matrix (basis matrix in the sparse representation), the second is to estimate sources (coefficient matrix). If the sources are sufficiently sparse, blind separation can be carried out directly in the time domain. Otherwise, blind separation can be implemented in time-frequency domain after applying wavelet packet transformation preprocessing to the observed mixtures. Three simulation examples are presented to illustrate the proposed algorithms and reveal algorithms performance. Finally, concluding remarks review the developed approach and state the open problems for further studying.

Application of spectral decomposition to compression and watermarking of 3D triangle mesh geometry

Abstract: Spectral decomposition of mesh geometry has been introduced by Taubin for geometry processing purposes. It has been extended to address transmission issues by Karni and Gotsman. Such a decomposition gives rise to pseudo-frequential information of the geometry defined over the mesh connectivity. For large meshes a piecewise decomposition has to be applied in order to restrict the complexity of the transform. In this paper, we propose to introduce overlap for its spectral representation. We show gains obtained in compression, progressive transmission and watermarking of mesh geometry. (C) 2003 Elsevier Science B.V. All rights reserved.

Array decomposition method for the accurate analysis of finite arrays

Abstract: Presented in this paper is a fast method to accurately model finite arrays of arbitrary three-dimensional elements. The proposed technique, referred to as the array decomposition method (ADM), exploits the repeating features of finite arrays and the free-space Green's function to assemble a nonsymmetric block-Toeplitz matrix system. The Toeplitz property is used to significantly reduce storage requirements and allows the fast Fourier transform (FFT) to be applied in accelerating the matrix-vector product operations of the iterative solution process. Each element of the array is modeled using the finite element-boundary integral (FE-BI) technique for rigorous analysis. Consequently, we demonstrate that the complete LU decomposition of the matrix system from a single array element can be used as a highly effective block-diagonal preconditioner on the larger array matrix system. This rigorous method is compared to the standard FE-BI technique for several tapered-slot antenna (TSA) arrays and is demonstrated to generate the same accuracy with a fraction of the storage and solution time.

An Overview of Matrix Factorization Theory and Operator Applications

Abstract: These lecture notes present an extensive review of the factorization theory of matrix functions relative to a curve with emphasis on the developments of the last 20–25 years. The classes of functions considered range from rational and continuous matrix functions to matrix functions with almost periodic or even semi almost periodic entries. Also included are recent results about explicit factorization based on the state space method from systems theory, with examples from linear transport theory. Related applications to Riemann-Hilbert boundary value problems and the Fredholm theory of various classes of singular integral operators are described too. The applications also concern inversion of singular integral operators of different types, including Wiener-Hopf and Toeplitz operators.

Recovery of constituent spectra using non-negative matrix factorization

Abstract: In this paper a constrained non-negative matrix factorization (cNMF) algorithm for recovering constituent spectra is described together with experiments demonstrating the broad utility of the approach. The algorithm is based on the NMF algorithm of Lee and Seung, extending it to include a constraint on the minimum amplitude of the recovered spectra. This constraint enables the algorithm to deal with observations having negative values by assuming they arise from the noise distribution. The cNMF algorithm does not explicitly enforce independence or sparsity, instead only requiring the source and mixing matrices to be non-negative. The algorithm is very fast compared to other "blind" methods for recovering spectra. cNMF can be viewed as a maximum likelihood approach for finding basis vectors in a bounded subspace. In this case the optimal basis vectors are the ones that envelope the observed data with a minimum deviation from the boundaries. Results for Raman spectral data, hyperspectral images, and 31P human brain data are provided to illustrate the algorithm's performance.

M-channel lifting factorization of perfect reconstruction filter banks and reversible M-band wavelet transforms

Abstract: An intrinsic M-channel lifting factorization of perfect reconstruction filter banks (PRFBs) is presented as an extension of Sweldens' conventional two-channel lifting scheme. Given a polyphase matrix E(z) of a finite-impulse response (FIR) M- channel PRFB with det(E(z))=z/sup -K/, K/spl isin//spl Zopf/, a systematic M-channel lifting factorization is derived based on the Monic Euclidean algorithm. The M-channel lifting structure provides an efficient factorization and implementation; examples include optimizing the factorization for the number of lifting steps, delay elements, and dyadic coefficients. Specialization to paraunitary building blocks enables the design of paraunitary filter banks based on lifting. We show how to achieve reversible, possibly multiplierless, implementations under finite precision, through the unit diagonal scaling property of the Monic Euclidean algorithm. Furthermore, filter-bank regularity of a desired order can be imposed on the lifting structure, and PRFBs with a prescribed admissible scaling filter are conveniently parameterized.

Multivariate Curve Resolution in the Analysis of Vibrational Spectroscopy Data Files

Abstract: T he ability of chemometric techniques to reduce and resolve large data matrices into responses and corresponding weights of these responses has been shown with a variety of data sets, and there exist a variety of approaches capable of assessing such bilinear data sets measured with vibrational spectroscopy.1–4 We have focused on using a form of principal factor analysis, which uses matrix decomposition as the initial step to generate abstract factors and scores. This initial estimate is then reŽ ned in a second step to produce spectra and intensity matrices. The second step uses an iterative constrained least-squares method referred to as alternating least squares. The technique, which is particularly useful in extracting estimates of pure analyte spectra and concentration proŽ les from spectral data matrices, is generally referred to

Scaled and decoupled Cholesky and QR decompositions with application to spherical MIMO detection

Abstract: Motivated by the need for the Cholesky factorization in implementing a spherical MIMO detector, this paper considers Cholesky and QR decompositions suitable for fixed-point implementation. In particular, we reformulate the decompositions to avoid the many square-root and division operations required in their natural form. This is achieved by decoupling the numerator and denominator calculations and applying scaling by powers of 2 (corresponding to bit shifts) to ensure numerical stability in the recursions. We consider the impact on the spherical detector formulation.