Showing papers on "Square matrix published in 2017"

Knowledge graph completion via complex tensor factorization

Abstract: In statistical relational learning, knowledge graph completion deals with automatically understanding the structure of large knowledge graphs--labeled directed graphs-- and predicting missing relationships--labeled edges. State-of-the-art embedding models propose different trade-offs between modeling expressiveness, and time and space complexity. We reconcile both expressiveness and complexity through the use of complex-valued embeddings and explore the link between such complex-valued embeddings and unitary diagonalization. We corroborate our approach theoretically and show that all real square matrices--thus all possible relation/adjacency matrices--are the real part of some unitarily diagonalizable matrix. This results opens the door to a lot of other applications of square matrices factorization. Our approach based on complex embeddings is arguably simple, as it only involves a Hermitian dot product, the complex counterpart of the standard dot product between real vectors, whereas other methods resort to more and more complicated composition functions to increase their expressiveness. The proposed complex embeddings are scalable to large data sets as it remains linear in both space and time, while consistently outperforming alternative approaches on standard link prediction benchmarks.

Revisiting the core EP inverse and its extension to rectangular matrices

Abstract: In this paper, we revise the core EP inverse of a square matrix introduced by Prasad and Mohana in [12], Core EP inverse, Linear and Multilinear Algebra 62 (3) (2014), 792-802. Firstly, we give a new representation and a new characterization of the core EP inverse. Then, we study some properties of the core EP inverse by using a representation by block matrices. Secondly, we extend the notion of core EP inverse to rectangular matrices by means of a weighted core EP decomposition. Finally, we study some properties of weighted core EP inverses. Keywords: Generalized inverses, Moore-Penrose inverse, core inverse, generalized core inverse, core EP inverse, DMP generalized inverse

Multivariate Statistical Data Analysis- Principal Component Analysis (PCA) -

Abstract: Principal component analysis (PCA) is a multivariate technique that analyzes a data table in which observations are described by several inter-correlated quantitative dependent variables. Its goal is to extract the important information from the statistical data to represent it as a set of new orthogonal variables called principal components, and to display the pattern of similarity between the observations and of the variables as points in spot maps. Mathematically, PCA depends upon the eigen-decomposition of positive semi-definite matrices and upon the singular value decomposition (SVD) of rectangular matrices. It is determined by eigenvectors and eigenvalues. Eigenvectors and eigenvalues are numbers and vectors associated to square matrices. Together they provide the eigen-decomposition of a matrix, which analyzes the structure of this matrix. Even though the eigen-decomposition does not exist for all square matrices, it has a particularly simple expression for matrices such as correlation, covariance, or cross-product matrices.

Approximating Spectral Sums of Large-Scale Matrices using Stochastic Chebyshev Approximations

Abstract: Computation of the trace of a matrix function plays an important role in many scientific computing applications, including applications in machine learning, computational physics (e.g., lattice qua...

Five-Dimensional Seismic Reconstruction Using Parallel Square Matrix Factorization

Abstract: Seismic data acquired by geophones are processed to estimate images of the earth’s interior that are used to explore, develop, and monitor resources and to study the shallow structure of the crust for geological, environmental, and geotechnical purposes. These multidimensional data sets are often irregularly sampled and incomplete in the so-called midpoint and offset acquisition coordinates. Multidimensional seismic data reconstruction can be viewed as a low-rank matrix or tensor completion problem. In this paper, we introduce a fast and efficient low-rank tensor completion algorithm named parallel square matrix factorization (PSMF) and adopt it to reconstruct seismic data in the typical seismic data processing coordinates: frequency, midpoint, and offset. For each frequency slice, we establish a tensor minimization model composed of a low-rank constrained term and a data misfit term. Then we adopt the PSMF algorithm for the recovery of the missing samples. In the PSMF method, we avoid using unbalanced “long strip” matrices that result from conventional tensor unfolding. Instead, the tensor is unfolded into almost square or square matrices that are low rank. We also compare the proposed PSMF method with other completion methods. Experiments via synthetic data and field data sets validate the effectiveness of the proposed algorithm.

Single-frame image super-resolution based on singular square matrix operator

Abstract: In the paper the method of single-frame image super-resolution based on the singular decomposition of matrix operator of the convergence square matrix operator is proposed. The characteristic vectors-features are obtained by using the Moore-Penrose pseudoinverse of matrix operator, which are used for enlarged image synthesis. The series of computational experiments based on images with fluctuation of intensity function are performed. The comparison results with others methods have confirmed the effectiveness of developed approach. The main advantages of proposed method for different enlargement coefficients are considered.

Theoretical properties of the global optimizer of two-layer Neural Network

Abstract: In this paper, we study the problem of optimizing a two-layer artificial neural network that best fits a training dataset. We look at this problem in the setting where the number of parameters is greater than the number of sampled points. We show that for a wide class of differentiable activation functions (this class involves "almost" all functions which are not piecewise linear), we have that first-order optimal solutions satisfy global optimality provided the hidden layer is non-singular. Our results are easily extended to hidden layers given by a flat matrix from that of a square matrix. Results are applicable even if network has more than one hidden layer provided all hidden layers satisfy non-singularity, all activations are from the given "good" class of differentiable functions and optimization is only with respect to the last hidden layer. We also study the smoothness properties of the objective function and show that it is actually Lipschitz smooth, i.e., its gradients do not change sharply. We use smoothness properties to guarantee asymptotic convergence of O(1/number of iterations) to a first-order optimal solution. We also show that our algorithm will maintain non-singularity of hidden layer for any finite number of iterations.

Algorithmic Regularization in Over-parameterized Matrix Recovery

Abstract: We study the problem of recovering a low-rank matrix $X^\star$ from linear measurements using an over-parameterized model. We parameterize the rank-$r$ matrix $X^\star$ by $UU^\top$ where $U\in \mathbb{R}^{d\times d}$ is a square matrix, whereas the number of linear measurements is much less than $d^2$. We show that with $\tilde{O}(dr^{2})$ random linear measurements, the gradient descent on the squared loss, starting from a small initialization, recovers $X^\star$ approximately in $\tilde{O}(\sqrt{r})$ iterations. The results solve the conjecture of Gunasekar et al. under the restricted isometry property, and demonstrate that the training algorithm can provide an implicit regularization for non-linear matrix factorization models.

The DMP inverse for rectangular matrices

Abstract: The definition of the DMP inverse of a square matrix with complex elements is extended to rectangular matrices by showing that for any A and W, m by n and n by m, respectively, there exists a unique matrix X, such that XAX = X, XA = Wad, wWA and (WA)k+1X =(WA)k+1A+, where Ad,w denotes the W-weighted Drazin inverse of A and k = Ind(AW), the index of AW.

A Formula for the Fréchet Derivative of a Generalized Matrix Function

Abstract: We state and prove an analogue of the Daleckii--Krein theorem, thus obtaining an explicit formula for the Frechet derivative of generalized matrix functions Moreover, we prove the differentiability of generalized matrix functions of real matrices under very mild assumptions For complex matrices, we argue that, under the same assumptions, generalized matrix functions are real-differentiable but generally not complex-differentiable Finally, we discuss the application of our results to the study of the condition number of generalized matrix functions Along our way, we also derive generalized matrix functional analogues of a few classical theorems on polynomial interpolation of classical matrix functions and their derivatives

Matrix Stabilization Using Differential Equations

Abstract: We consider the problem of stabilizing a matrix by a correction of minimal norm: Given a square matrix that has some eigenvalues with positive real part, find the nearest matrix having no eigenvalu...

Increasing the Efficiency of Sparse Matrix-Matrix Multiplication with a 2.5D Algorithm and One-Sided MPI

Abstract: Matrix-matrix multiplication is a basic operation in linear algebra and an essential building block for a wide range of algorithms in various scientific fields. Theory and implementation for the dense, square matrix case are well-developed. If matrices are sparse, with application-specific sparsity patterns, the optimal implementation remains an open question. Here, we explore the performance of communication reducing 2.5D algorithms and one-sided MPI communication in the context of linear scaling electronic structure theory. In particular, we extend the DBCSR sparse matrix library, which is the basic building block for linear scaling electronic structure theory and low scaling correlated methods in CP2K. The library is specifically designed to efficiently perform block-sparse matrix-matrix multiplication of matrices with a relatively large occupation. Here, we compare the performance of the original implementation based on Cannon's algorithm and MPI point-to-point communication, with an implementation based on MPI one-sided communications (RMA), in both a 2D and a 2.5D approach. The 2.5D approach trades memory and auxiliary operations for reduced communication, which can lead to a speedup if communication is dominant. The 2.5D algorithm is somewhat easier to implement with one-sided communications. A detailed description of the implementation is provided, also for non ideal processor topologies, since this is important for actual applications. Given the importance of the precise sparsity pattern, and even the actual matrix data, which decides the effective fill-in upon multiplication, the tests are performed within the CP2K package with application benchmarks. Results show a substantial boost in performance for the RMA based 2.5D algorithm, up to 1.80x, which is observed to increase with the number of processes involved in the parallelization.

Polynomial Ensembles and P\'olya Frequency Functions

Abstract: We study several kinds of polynomial ensembles of derivative type which we propose to call Polya ensembles. These ensembles are defined on the spaces of complex square, complex rectangular, Hermitian, Hermitian anti-symmetric and Hermitian anti-self-dual matrices, and they have nice closure properties under the multiplicative convolution for the first class and under the additive convolution for the other classes. The cases of complex square matrices and Hermitian matrices were already studied in former works. One of our goals is to unify and generalize the ideas to the other classes of matrices. Here we consider convolutions within the same class of Polya ensembles as well as convolutions with the more general class of polynomial ensembles. Moreover, we derive some general identities for group integrals similar to the Harish-Chandra-Itzykson-Zuber integral, and we relate Polya ensembles to Polya frequency functions. For illustration we give a number of explicit examples for our results.

Computing nearest stable matrix pairs

Abstract: In this paper, we study the nearest stable matrix pair problem: given a square matrix pair $(E,A)$, minimize the Frobenius norm of $(\Delta_E,\Delta_A)$ such that $(E+\Delta_E,A+\Delta_A)$ is a stable matrix pair. We propose a reformulation of the problem with a simpler feasible set by introducing dissipative Hamiltonian (DH) matrix pairs: A matrix pair $(E,A)$ is DH if $A=(J-R)Q$ with skew-symmetric $J$, positive semidefinite $R$, and an invertible $Q$ such that $Q^TE$ is positive semidefinite. This reformulation has a convex feasible domain onto which it is easy to project. This allows us to employ a fast gradient method to obtain a nearby stable approximation of a given matrix pair.

On the incomplete hypergeometric matrix functions

Abstract: By means of the familiar incomplete gamma matrix functions \(\gamma (A,x)\) and \(\Gamma (A,x)\), we introduce the incomplete Pochhammer matrix symbols that lead us to a generalization and decomposition of the incomplete hypergeometric matrix functions (IHMFs). Some properties such as a matrix differential equation, integral expressions and recurrence relations of IHMFs are given. Besides, connections between these matrix functions and other special matrix functions are investigated.

On the connection between discrete linear repetitive processes and 2-D discrete linear systems

Abstract: A direct method is developed that reduces a polynomial system matrix describing a discrete linear repetitive process to a 2-D singular state-space form such that all the relevant properties, including the zero structure of the system matrix, are retained. It is shown that the transformation linking the original polynomial system matrix with its associated 2-D singular form is zero coprime system equivalence. The exact nature of the resulting system matrix in singular form and the transformation involved are established.

Knowledge Graph Completion via Complex Tensor Factorization

Abstract: In statistical relational learning, knowledge graph completion deals with automatically understanding the structure of large knowledge graphs---labeled directed graphs---and predicting missing relationships---labeled edges. State-of-the-art embedding models propose different trade-offs between modeling expressiveness, and time and space complexity. We reconcile both expressiveness and complexity through the use of complex-valued embeddings and explore the link between such complex-valued embeddings and unitary diagonalization. We corroborate our approach theoretically and show that all real square matrices---thus all possible relation/adjacency matrices---are the real part of some unitarily diagonalizable matrix. This results opens the door to a lot of other applications of square matrices factorization. Our approach based on complex embeddings is arguably simple, as it only involves a Hermitian dot product, the complex counterpart of the standard dot product between real vectors, whereas other methods resort to more and more complicated composition functions to increase their expressiveness. The proposed complex embeddings are scalable to large data sets as it remains linear in both space and time, while consistently outperforming alternative approaches on standard link prediction benchmarks.

Matrix balancing in Lp norms: bounding the convergence rate of osborne's iteration

Abstract: We study an iterative matrix conditioning algorithm due to Osborne (1960). The goal of the algorithm is to convert a square matrix into a balanced matrix where every row and corresponding column have the same norm. The original algorithm was proposed for balancing rows and columns in the L2 norm, and it works by iterating over balancing a row-column pair in fixed round-robin order. Variants of the algorithm for other norms have been heavily studied and are implemented as standard preconditioners in many numerical linear algebra packages. Recently, Schulman and Sinclair (2015), in a first result of its kind for any norm, analyzed the rate of convergence of a variant of Osborne's algorithm that uses the L∞ norm and a different order of choosing row-column pairs. In this paper we study matrix balancing in the L1 norm and other Lp norms. We show the following results for any matrix A = (aij)ni,j=1, resolving in particular a main open problem mentioned by Schulman and Sinclair.1. We analyze the iteration for the L1 norm under a greedy order of balancing. We show that it converges to an ϵ-balanced matrix in K = O(min{ϵ−2 log w, ϵ−1n3/2 log(w/ϵ)}) iterations that cost a total of O(m + K n log n) arithmetic operations over O(n log(w/ϵ))-bit numbers. Here m is the number of non-zero entries of A, and w = Σi,j |ai,j|/amin with amin = min{|aij| : aij ≠ 0}.2. We show that the original round-robin implementation converges to an ϵ-balanced matrix in O(ϵ−2n2 log w) iterations totaling O(ϵ−2mn log w) arithmetic operations over O(n log(w/ϵ))-bit numbers.3. We show that a random implementation of the iteration converges to an ϵ-balanced matrix in O(ϵ−2 log w) iterations using O(m + ϵ−2n log w) arithmetic operations over O(log(wn/ϵ))-bit numbers.4. We demonstrate a lower bound of [EQUATION] on the convergence rate of any implementation of the iteration.5. We observe, through a known trivial reduction, that our results for L1 balancing apply to any Lp norm for all finite p, at the cost of increasing the number of iterations by only a factor of p.We note that our techniques are very different from those used by Schulman and Sinclair.

Improved Rectangular Matrix Multiplication using Powers of the Coppersmith-Winograd Tensor

Abstract: In the past few years, successive improvements of the asymptotic complexity of square matrix multiplication have been obtained by developing novel methods to analyze the powers of the Coppersmith-Winograd tensor, a basic construction introduced thirty years ago. In this paper we show how to generalize this approach to make progress on the complexity of rectangular matrix multiplication as well, by developing a framework to analyze powers of tensors in an asymmetric way. By applying this methodology to the fourth power of the Coppersmith-Winograd tensor, we succeed in improving the complexity of rectangular matrix multiplication. Let $\alpha$ denote the maximum value such that the product of an $n\times n^\alpha$ matrix by an $n^\alpha\times n$ matrix can be computed with $O(n^{2+\epsilon})$ arithmetic operations for any $\epsilon>0$. By analyzing the fourth power of the Coppersmith-Winograd tensor using our methods, we obtain the new lower bound $\alpha>0.31389$, which improves the previous lower bound $\alpha>0.30298$ obtained five years ago by Le Gall (FOCS'12) from the analysis of the second power of the Coppersmith-Winograd tensor. More generally, we give faster algorithms computing the product of an $n\times n^k$ matrix by an $n^k\times n$ matrix for any value $k eq 1$. (In the case $k=1$, we recover the bounds recently obtained for square matrix multiplication). These improvements immediately lead to improvements in the complexity of a multitude of fundamental problems for which the bottleneck is rectangular matrix multiplication, such as computing the all-pair shortest paths in directed graphs with bounded weights.

On the efficient computation of a generalized Jacobian of the projector over the Birkhoff polytope.

Abstract: We derive an explicit formula, as well as an efficient procedure, for constructing a generalized Jacobian for the projector of a given square matrix onto the Birkhoff polytope, i.e., the set of doubly stochastic matrices. To guarantee the high efficiency of our procedure, a semismooth Newton method for solving the dual of the projection problem is proposed and efficiently implemented. Extensive numerical experiments are presented to demonstrate the merits and effectiveness of our method by comparing its performance against other powerful solvers such as the commercial software Gurobi and the academic code PPROJ [{\sc Hager and Zhang}, SIAM Journal on Optimization, 26 (2016), pp.~1773--1798]. In particular, our algorithm is able to solve the projection problem with over one billion variables and nonnegative constraints to a very high accuracy in less than 15 minutes on a modest desktop computer. More importantly, based on our efficient computation of the projections and their generalized Jacobians, we can design a highly efficient augmented Lagrangian method (ALM) for solving a class of convex quadratic programming (QP) problems constrained by the Birkhoff polytope. The resulted ALM is demonstrated to be much more efficient than Gurobi in solving a collection of QP problems arising from the relaxation of quadratic assignment problems.

N-dimensional Rotation Matrix Generation Algorithm

Abstract: This article presents a new algorithm for generation of N-dimensional rotation matrix M, which rotates given N-dimensional vector X to the direction of given vector Y which has the same dimension. Algorithm, named N-dimensional Rotation Matrix Generation Algorithm (NRMG) includes rotation of given vectors X and Y to the direction of coordinate axis x1 using two-dimensional rotations. Matrix M is obtained as multiplication of matrix MX and inverse of matrix MY, which rotates given vectors to the direction of axis x1. Also examined is the possibility to perform parallel calculations of two-dimensional rotations.

A Fast Algorithm to Estimate Phasor in Power Systems

Abstract: This paper proposes a fast phasor estimation method for power systems. Using the matrix pencil method, the multiplication of the pseudoinverse of the sampling matrix and the reference signal matrix constructs a square matrix. The fundamental sinusoidal component of the signal is derived by finding the eigenvalues of this square matrix. QR factorization and similarity transformation are introduced to compute the eigenvalues of the fundamental component by reducing the matrix order to two. The proposed method is compared with recently proposed methods, including the Fourier algorithm and least-square method. Extensive simulation results demonstrate that, with only half-cycle sampling data, the presented method has low computational complexity and its high accuracy is not affected by harmonics or decaying dc component even during the transient process.

On the Yang-Baxter-like matrix equation for rank-two matrices

Abstract: Abstract Let A = PQT, where P and Q are two n × 2 complex matrices of full column rank such that QTP is singular. We solve the quadratic matrix equation AXA = XAX. Together with a previous paper devoted to the case that QTP is nonsingular, we have completely solved the matrix equation with any given matrix A of rank-two.