Showing papers on "QR decomposition published in 2020"

Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares [Bookshelf]

Abstract: The book consists of three parts. Part 1 focuses on vectors and their manipulation. Vector algebra, linear functions, linearization, inner products, norms, linear independence, the concept of a basis, and orthogonality are covered. Part 2 is devoted to matrices. Basic matrix operations (including a clear discussion of all four ways to interpret matrix multiplication), linear systems of equations, the QR factorization, and various matrix inverses (left, right, and pseudoinverses) are discussed. Part 3 covers least squares and emphasizes different approaches to finding xt. A notable feature of this book is the excellent end-ofchapter problem sets that span a wide range of interesting applications. A detailed solutions manual is also available to instructors. The book is clear and direct, written by experienced authors. The need for textbooks that show mathematics in action is also clear. Linear algebra has become more important to modern students (arguably, more important than calculus), partly because of the rise of data science and deep learning. Through it all, this wonderful subject retains the beauty of good mathematics.

Serverless linear algebra

Abstract: Datacenter disaggregation provides numerous benefits to both the datacenter operator and the application designer. However switching from the server-centric model to a disaggregated model requires developing new programming abstractions that can achieve high performance while benefiting from the greater elasticity. To explore the limits of datacenter disaggregation, we study an application area that near-maximally benefits from current server-centric datacenters: dense linear algebra. We build NumPyWren, a system for linear algebra built on a disaggregated serverless programming model, and LAmbdaPACK, a companion domain-specific language designed for serverless execution of highly parallel linear algebra algorithms. We show that, for a number of linear algebra algorithms such as matrix multiply, singular value decomposition, Cholesky decomposition, and QR decomposition, NumPyWren's performance (completion time) is within a factor of 2 of optimized server-centric MPI implementations, and has up to 15% greater compute efficiency (total CPU-hours), while providing fault tolerance.

Double image encryption using 3D Lorenz chaotic system, 2D non-separable linear canonical transform and QR decomposition

Abstract: In this paper, a double image encryption scheme using 3D Lorenz chaotic system and QR decomposition in 2D non-separable linear canonical transform domain is proposed. Here the independent parameters of the 2D non-separable canonical transform enlarge the key-space of the proposed scheme and enhance robustness against brute-force attack. 3D Lorenz chaotic system is employed for creating a permutation keystream for pixel transaction process. The proposed scheme is non-linear and asymmetric in nature. To validate and verify the proposed cryptosystem, the numerical simulations have been performed on grayscale images. Results display that the proposed scheme has greater robustness to occlusion and special attacks.

Patient data hiding into ECG signal using watermarking in transform domain

Abstract: Electrocardiogram (ECG) watermarking provides secure communication of patient information lies in a 1D- ECG signal. The primary challenge in ECG watermarking is the deterioration of an ECG signal which causes the loss and impotence to extract patient information. This paper proposes a wavelet method based watermarking scheme for patient information hiding in the ECG as a QR image. Here, we first convert the 1D-ECG signal to 2D-ECG image using the Pan–Tompkins algorithm. We use a wavelet transform to decompose 2D-ECG image. Wavelet analysis can capture the subtle underlying information of the ECG. Then we further decompose the detail coefficient of wavelet and the QR image using QR decomposition for embedding data. The embedding factor value calculation is adaptive by harnessing the entropy value of the signal. The hidden data is easily extractable with no distortion at the extractor side. The ECG data we use in this paper is from the MIT-BIH database. The results on this dataset suggest that our proposed approach is useful in patient information data hiding scheme in ECG. The proposed method outperforms the state-of-the-art.

A hybrid symmetry–PSO approach to finding the self-equilibrium configurations of prestressable pin-jointed assemblies

Abstract: For pin-jointed assemblies with many members or self-stress states, the form-finding problem using conventional methods generally involves considerable computational complexities due to the large size of the solution spaces. Here, we propose an improved form-finding method for prestressable pin-jointed structures by combining symmetry-based qualitative analysis with particle swarm optimization. Expressed in the symmetry-adapted coordinate system, the nodal coordinate vectors of a structure with specific symmetry and topology are independently extracted from the key blocks of the small-sized force density matrices associated with rigid-body translations. Then, the first block of the equilibrium matrix is computed, in which the null space reveals integral self-stress states. Particle swarm optimization is introduced and adapted to find feasible prestress modes, where the uniformity and unilaterality conditions of the members are considered. Besides, the QR decomposition with column pivoting is adopted for efficient computations on the null space of these blocks. The QR decompositions of the small-sized blocks of the force density matrix and the equilibrium matrix are performed iteratively, to simultaneously find a stable self-equilibrium configuration and a feasible prestress mode. Representative examples show the presented method is computationally efficient and accurate for the form-finding of symmetric tensegrities and prestressed cable–strut structures.

Machine Learning K-Means Clustering Algorithm for Interpolative Separable Density Fitting to Accelerate Hybrid Functional Calculations with Numerical Atomic Orbitals.

Abstract: The interpolative separable density fitting (ISDF) is an efficient and accurate low-rank decomposition method to reduce the high computational cost and memory usage of the Hartree-Fock exchange (HFX) calculations with numerical atomic orbitals (NAOs). In this work, we present a machine learning K-means clustering algorithm to select the interpolation points in ISDF, which offers a much cheaper alternative to the expensive QR factorization with column pivoting (QRCP) procedure. We implement this K-means-based ISDF decomposition to accelerate hybrid functional calculations with NAOs in the HONPAS package. We demonstrate that this method can yield a similar accuracy for both molecules and solids at a much lower computational cost. In particular, K-means can remarkably reduce the computational cost of selecting the interpolation points by nearly two orders of magnitude compared to QRCP, resulting in a speedup of ∼10 times for ISDF-based HFX calculations.

Sensor Selection With Cost Constraints for Dynamically Relevant Bases

Abstract: We consider cost-constrained sparse sensor selection for full-state reconstruction, applying a well-known greedy algorithm to dynamical systems for which the usual singular value decomposition (SVD) basis may not be available or preferred. We apply the cost-modified, column-pivoted QR decomposition to a physically relevant basis—the pivots correspond to sensor locations, and these locations are penalized with a heterogeneous cost function. In considering different bases, we are able to account for the dynamics of the particular system, yielding sensor arrays that are nearly Pareto optimal in sensor cost and performance in the chosen basis. This flexibility extends our framework to include actuation and dynamic estimation, and to select sensors without training data. We provide three examples from the physical and engineering sciences and evaluate sensor selection in three dynamically relevant bases: truncated balanced modes for control systems, dynamic mode decomposition (DMD) modes, and a basis of analytic modes. We find that these bases all yield effective sensor arrays and reconstructions for their respective systems. When possible, we compare to results using an SVD basis and evaluate tradeoffs between methods.

Near-Optimal MIMO-SCMA Uplink Detection With Low-Complexity Expectation Propagation

Abstract: Multiple-input multiple-output (MIMO) and sparse code multiple access (SCMA) can be combined to achieve higher spectrum efficiency and more access for users, which also introduces more difficulties in signal detection. This paper explores low-complexity and low-latency iterative algorithms for soft symbol detection in an uplink MIMO-SCMA system over Rayleigh flat-fading channels. An expectation propagation framework (EPA) based on the extended factor graph is developed for MIMO-SCMA with multiantenna users. A new initialization method is proposed to accelerate convergence. Moreover, the SC-EPA with lower complexity is proposed by introducing QR decomposition and RE cluster-based decentralized factor node (FN) processing. Furthermore, new approaches for message passing between variable nodes (VNs) and FNs are proposed to improve the parallelism and reduce the complexity of the algorithm. The complexity of SC-EPA scales linearly with constellation size $\Omega $ ( $\Omega ) and is independent of the receiving antenna $\text {N}_{\text {r}}$ without any performance penalties. The robustness of the proposed algorithm in imperfect channels is evaluated, and the state evolution (SE) of the SC-EPA is derived. The link-level simulation results demonstrate that the EPA and SC-EPA receivers can achieve nearly the same performance as state of-the-art methods but with much lower complexity.

Computation of outer inverses of tensors using the QR decomposition

Abstract: In this paper, we introduce new representations and characterizations of the outer inverse of tensors through QR decomposition Derived representations are usable in generating corresponding representations of main tensor generalized inverses Some results on reshape operation of a tensor are added to the existing theory An effective algorithm for computing outer inverses of tensors is proposed and applied The power of the proposed method is demonstrated by its application in 3D color image deblurring

Efficient QR Updating Factorization for Sensorless Synchronous Motor Drive Based on High Frequency Voltage Injection

Abstract: The conventional methods for estimating the rotor position of permanent magnet synchronous machines, at low speed range and characterized by rotor saliency, rely on high frequency voltage injection in the stator windings. Ordinarily, the rotor position estimation is achieved through the demodulation of the high frequency current response. In this article, an alternative method is presented for detecting rotor position from the rotating high frequency injection current response. The proposed ellipse fitting technique is not affected by signal processing delay effects and it requires the tuning of only one parameter, called forgetting factor, making the studied method suitable for industrial application thanks to its minimal setup effort. The inverse problem related to the ellipse fitting is solved implementing a QR recursive least squares algorithm. Efficient updating QR factorization has been adopted because of its features in terms of numerical stability and required limited computational effort. The proposed sensorless control scheme is validated by means of many experiments.

Randomized Gram-Schmidt process with application to GMRES.

Abstract: A randomized Gram-Schmidt algorithm is developed for orthonormalization of high-dimensional vectors or QR factorization. The proposed process can be less computationally expensive than the classical Gram-Schmidt process while being at least as numerically stable as the modified Gram-Schmidt process. Our approach is based on random sketching, which is a dimension reduction technique consisting in estimation of inner products of high-dimensional vectors by inner products of their small efficiently-computable random projections, so-called sketches. This allows to perform the projection step in Gram-Schmidt process on sketches rather than high-dimensional vectors with a minor computational cost. This also provides an ability to efficiently certify the output. The proposed Gram-Schmidt algorithm can provide computational cost reduction in any architecture. The benefit of random sketching can be amplified by exploiting multi-precision arithmetic. We provide stability analysis for multi-precision model with coarse unit roundoff for standard high-dimensional operations. Numerical stability is proven for the unit roundoff independent of the (high) dimension of the problem. The proposed Gram-Schmidt process can be applied to Arnoldi iteration and result in new Krylov subspace methods for solving high-dimensional systems of equations or eigenvalue problems. Among them we chose randomized GMRES method as a practical application of the methodology.

Shifted Cholesky QR for Computing the QR Factorization of Ill-Conditioned Matrices

Abstract: The Cholesky QR algorithm is an efficient communication-minimizing algorithm for computing the QR factorization of a tall-skinny matrix $X\in\mathbb{R}^{m\times n}$, where $m\gg n$. Unfortunately i...

A Bounded-Variable Least-Squares Solver Based on Stable QR Updates

Abstract: In this paper, a numerically robust solver for least-square problems with bounded variables (BVLS) is presented for applications including, but not limited to, model predictive control (MPC). The proposed BVLS algorithm solves the problem efficiently by employing a recursive QR factorization method based on Gram–Schmidt orthogonalization. A reorthogonalization procedure that iteratively refines the QR factors provides numerical robustness for the described primal active-set method, which solves a system of linear equations in each of its iterations via recursive updates. The performance of the proposed BVLS solver, which is implemented in C without external software libraries, is compared in terms of computational efficiency against state-of-the-art quadratic programming solvers for small- to medium-sized random BVLS problems and a typical example of embedded linear MPC application. The numerical tests demonstrate that the solver performs very well even when solving ill-conditioned problems in single-precision floating-point arithmetic.

Quaternion Discrete Fourier Transform-Based Color Image Watermarking Method Using Quaternion QR Decomposition

Abstract: In this paper, a new Quaternion Discrete Fourier Transform (QDFT)-based digital color image watermarking method is presented. In addition, the Quaternion QR (QQR) decomposition is applied in digital watermarking technology for the first time. First of all, the QDFT and QQR decomposition are performed on the host image, respectively, to acquire the scalar part of the quaternion matrix for watermark information embedding. After that, we divide the scalar part of the quaternion matrix generated by the QQR decomposition into blocks and calculate the entropy. The block with high entropy is selected to embed the watermark information. Then the watermark information is embedded into the extracted block using the quantization index modulation method. We conducted a large number of tests and experimental results indicate that the presented approach obtains excellent robustness against Scaling, Rotation, Median filtering, `Salt & Pepper' noise, and JPEG Compression. Compared with the existing methods, the presented method achieves better performance.

A Real-Time Capable Forward Kinematics Algorithm for Cable-Driven Parallel Robots Considering Pulley Kinematics

Abstract: A real-time capable Forward Kinematics (FK) algorithm for Cable-Driven Parallel Robots (CDPRs) considering the pulley kinematics is proposed. The algorithm applies iteratively QR decomposition to solve a linearized version of the least squares problem representing the FK. Differential kinematics delivers an analytical expression for the Jacobian matrix of CDPRs considering the pulley kinematics. This Jacobian matrix is used to construct the linearization of the FK problem. Experimental and numerical results address the convergence capabilities of the proposed algorithm.

Fast and Robust Low-Rank Approximation for Five-Dimensional Seismic Data Reconstruction

Abstract: Five-dimensional (5D) seismic data reconstruction becomes more appealing in recent years because it takes advantage of five physical dimensions of the seismic data and can reconstruct data with large gap. The low-rank approximation approach is one of the most effective methods for reconstructing 5D dataset. However, the main disadvantage of the low-rank approximation method is its low computational efficiency because of many singular value decompositions (SVD) of the block Hankel/Toeplitz matrix in the frequency domain. In this paper, we develop an SVD-free low-rank approximation method for efficient and effective reconstruction and denoising of the seismic data that contain four spatial dimensions. Our SVD-free rank constraint model is based on an alternating minimization strategy, which updates one variable each time while fixing the other two. For each update, we only need to solve a linear least-squares problem with much less expensive QR factorization. The SVD-based and SVD-free low-rank approximation methods in the singular spectrum analysis (SSA) framework are compared in detail, regarding the reconstruction performance and computational cost. The comparison shows that the SVD-free low-rank approximation method can obtain similar reconstruction performance as the SVD-based method but with a large computational speedup.

LU-Cholesky QR algorithms for thin QR decomposition

Abstract: This paper aims to propose the LU-Cholesky QR algorithms for thin QR decomposition (also called economy size or reduced QR decomposition). CholeskyQR is known as a fast algorithm employed for thin QR decomposition, and CholeskyQR2 aims to improve the orthogonality of a Q-factor computed by CholeskyQR. Although such Cholesky QR algorithms can efficiently be implemented in high-performance computing environments, they are not applicable for ill-conditioned matrices, as compared to the Householder QR and the Gram–Schmidt algorithms. To address this problem, we apply the concept of LU decomposition to the Cholesky QR algorithms, i.e., the idea is to use LU-factors of a given matrix as preconditioning before applying Cholesky decomposition. Moreover, we present rounding error analysis of the proposed algorithms on the orthogonality and residual of computed QR-factors. Numerical examples provided in this paper illustrate the efficiency of the proposed algorithms in parallel computing on both shared and distributed memory computers.

Randomized Algorithms for Generalized Singular Value Decomposition with Application to Sensitivity Analysis

Abstract: The generalized singular value decomposition (GSVD) is a valuable tool that has many applications in computational science. However, computing the GSVD for large-scale problems is challenging. Motivated by applications in hyper-differential sensitivity analysis (HDSA), we propose new randomized algorithms for computing the GSVD which use randomized subspace iteration and weighted QR factorization. Detailed error analysis is given which provides insight into the accuracy of the algorithms and the choice of the algorithmic parameters. We demonstrate the performance of our algorithms on test matrices and a large-scale model problem where HDSA is used to study subsurface flow.

Model Reduction for Multi-Scale Transport Problems using Structure-Preserving Least-Squares Projections with Variable Transformation

Abstract: A projection-based formulation is presented for non-linear model reduction of problems with extreme scale disparity. The approach allows for the selection of an arbitrary, but complete, set of solution variables while preserving the structure of the governing equations. Least-squares-based minimization is leveraged to guarantee symmetrization and discrete consistency with the full-order model (FOM) at the sub-iteration level. Two levels of scaling are used to achieve the conditioning required to effectively handle problems with extremely disparate physical phenomena, characterized by extreme stiffness in the system of equations. The formulation -- referred to as structure-preserving least-squares with variable transformation (SP-LSVT) -- provides global stabilization for both implicit and explicit time integration schemes. To achieve computational efficiency, a pivoted QR decomposition is used with oversampling, and adapted to the SP-LSVT method. The framework is demonstrated in representative two- and three-dimensional reacting flow problems, and the SP-LSVT is shown to exhibit improved stability and accuracy over standard projection-based ROM techniques. Physical realizability and local stability are promoted by enforcing limiters in both temperature and species mass fractions. These limiters are demonstrated to be important in eliminating regions of spurious burning, thus enabling the ROMs to provide accurate representations of the heat release rate and flame propagation speed. In the 3D application, it is shown that more than two orders of magnitude acceleration in computational efficiency can be achieved, while also providing reasonable future-state predictions. A key contribution of this work is the development and demonstration of a comprehensive ROM formulation that targets highly challenging multi-scale transport-dominated problems.

Randomized Projection for Rank-Revealing Matrix Factorizations and Low-Rank Approximations

Abstract: Rank-revealing matrix decompositions provide an essential tool in spectral analysis of matrices, including the Singular Value Decomposition (SVD) and related low-rank approximation techniques. QR w...

High Accuracy Matrix Computations on Neural Engines: A Study of QR Factorization and its Applications

Abstract: Fueled by the surge of ever expanding successful applications of deep neural networks and the great computational power demanded, modern computer processors and accelerators are beginning to offer half precision floating point arithmetic support, and special units (neural engines) such as NVIDIA TensorCore on GPU and Google Tensor Processing Unit (TPU) to accelerate the training and prediction of deep neural networks It remains unclear how neural engines can be profitably used in applications other than neural networks In this paper we present an endeavor of accelerating and stabilizing a fundamental matrix factorization on neural engines---the QR factorization---which may open doors to much wider relevance to scientific, engineering, and data science We show that traditional Householder QR algorithms and implementations do not have the necessary data locality, parallelism, accuracy, and robustness on neural engines which are characterized by extreme speed and low precision/range We demonstrate that neural engines can be effectively used to accelerate matrix computations (QR 30x-146x speedup compared to cuSOLVER, reaching up to 366TFLOPS); however different algorithms (recursive Gram-Schmidt) are needed to expose more locality and parallelism, even at the cost of increased computations Moreover, scaling, iterative refinement, and other safeguarding procedures are also needed to regain accuracy and avoid overflowing Our experience seems to suggest that presently with neural engines the matrix factorizations (QR, LU, Cholesky) are best to be co-designed with its applications (linear solver, least square, orthogonalization, SVD, etc) to achieve high performance and adequate accuracy and reliability

Primal-dual path-following methods and the trust-region updating strategy for linear programming with noisy data

Abstract: In this article, we consider the primal-dual path-following method and the trust-region updating strategy for the standard linear programming problem. For the rank-deficient problem with the small noisy data, we also give the preprocessing method based on the QR decomposition with column pivoting. Then, we prove the global convergence of the new method when the initial point is strictly primal-dual feasible. Finally, for some rank-deficient problems with or without the small noisy data from the NETLIB collection, we compare it with other two popular interior-point methods, i.e. the subroutine pathfollow.m and the built-in subroutine linprog.m of the MATLAB environment. Numerical results show that the new method is more robust than the other two methods for the rank-deficient problem with the small noise data.