Least squares approximation is a technique to find an approximate solution to a system of linear equations that has no exact solution. In a typical setting, one lets n be the number of constraints and d be the number of variables, with $${n \gg d}$$. Then, existing exact methods find a solution vector in O(nd 2) time. We present two randomized algorithms that provide accurate relative-error approximations to the optimal value and the solution vector of a least squares approximation problem more rapidly than existing exact algorithms. Both of our algorithms preprocess the data with the Randomized Hadamard transform. One then uniformly randomly samples constraints and solves the smaller problem on those constraints, and the other performs a sparse random projection and solves the smaller problem on those projected coordinates. In both cases, solving the smaller problem provides relative-error approximations, and, if n is sufficiently larger than d, the approximate solution can be computed in O(nd ln d) time.

Faster least squares approximation

Computing canonical forms of matrices over rings is a classical math¬ ematical problem with many applications to computational linear alge¬ bra. These forms include the Frobenius form over a field, the Hermite form over a principal ideal domain and the Howell and Smith form over a principal ideal ring. Generic algorithms are presented for computing each of these forms together with associated unimodular transformation matrices. The algorithms are analysed, with respect to the worst case, in terms of number of required operations from the ring. All algorithms are deterministic. For a square input matrix, the algorithms recover each of these forms in about the same number of operations as required for matrix multiplication. Special emphasis is placed on the efficient computation of transforms for the Hermite and Smith form in the case of rectangular input matrices. Here we analyse the running time of our algorithms in terms of three parameters: the row dimension, the column dimension and the number of nonzero rows in the output matrix. The generic algorithms are applied to the problem of computing the Hermite and Smith form of an integer matrix. Here the complexity anal¬ ysis is in terms of number of bit operations. Some additional techniques are developed to avoid intermediate expression swell. New algorithms are demonstrated to construct transformation matrices which have good bounds on the size of entries. These algorithms recover transforms in essentially the same time as required by our algorithms to compute only the form itself. Kurzfassung Kanonischen Formen von Matrizen über Ringen zu berechnen, ist ein klassisches mathematisches Problem mit vielen Anwendungen zur konstruktiven linearen Algebra. Diese Formen umfassen die Frobenius Form über einem Körper und die Hermite-, Howellund Smith-Form über einem Hauptidealring. Wir studieren die Berechnung dieser Formen aus der Sicht von sequentiellen deterministischen Komplexitätsschranken im schlimmsten Fall. Wir präsentieren Algorithmen für das Berechnen aller dieser Formen sowie der dazugehörigen unimodularen Transforma¬ tionsmatrizen samt Analyse der Anzahl benötigten Ringoperationen. Die Howell-, HermiteSmithund Frobenius-Form einer quadratischen Matrix kann mit ungefähr gleich vielen Operationen wie die Matrixmul¬ tiplikation berechnet werden. Ein Schwerpunkt liegt hier bei der effizienten Berechnung der Hermiteund Smith-Form sowie der dazugehörigen Transformationsmatrizen im Falle einer nichtquadratischen Eingabematrix. In diesem Fall analysieren wir die Laufzeit unserer Algorithmen abhänhig von drei Parametern: die Anzahl der Zeilen, die Anzahl der Spalten und die Anzahl der Zeilen in der berechneten Form, die mindestens ein Element ungleich Null enthal¬ ten. Die generische Algorithmen werden auf das Problem des Aufsteilens der Hermiteund Smith-Form einer ganzzahligen Matrix angewendet. Hier wird die Komplizität des Verfahren in der Anzahl der benötigten Bitoperationen ausgedrückt. Einige zusätzliche Techniken wurden en¬ twickelt, um das übermässige Wachsen von Zwischenergebnissen zu ver¬ meiden. Neue Verfahren zur Konstruktion von Transformationsmatrizen für die Hermiteund Smith-Form einer ganzzahligen Matrix wurden en¬ twickelt. Ziel der Bemühungen bei der Entwicklung dieser Verfahren war im Wesentlichen das erreichen der gleichen obere Schranke für die Laufzeit, die unsere Algorithmen benötigen, um nur die Form selbst zu berechnen.

Algorithms for matrix canonical forms

We study the link between the complexity of polynomial matrix multiplication and the complexity of solving other basic linear algebra problems on polynomial matrices. By polynomial matrices we mean ntimes n matrices in K[x] of degree bounded by d, with K a commutative field. Under the straight-line program model we show that multiplication is reducible to the problem of computing the coefficient of degree d of the determinant. Conversely, we propose algorithms for minimal approximant computation and column reduction that are based on polynomial matrix multiplication; for the determinant, the straight-line program we give also relies on matrix product over K[x] and provides an alternative to the determinant algorithm of [16, 17]. We further show that all these problems can be solved in particular in O(ω) operations in K. Here the "soft O" notation O indicates some missing log (nd) factors and ω is the exponent of matrix multiplication over K.

/pdf/on-the-complexity-of-polynomial-matrix-computations-4uxbl2nz6w.pdf

On the complexity of polynomial matrix computations

We present new algorithms for computing Smith normal forms of matrices over the integers and over the integers modulo d For the case of matrices over Z d we present an algorithm that computes the Smith form S of an A Z n m d in only O n m operations from Z d Here is the ex ponent for matrix multiplication over rings two n n matrices over a ring R can be multiplied in O n opera tions from R We apply our algorithm for matrices over Z d to get an algorithm for computing the Smith form S of an A Z n m in O n m M n log jjAjj bit opera tions where jjAjj max jAi jj and M t bounds the cost of multiplying two dte bit integers These complexity re sults improve signi cantly on the complexity of previously best known Smith form algorithms both deterministic and probabilistic which guarantee correctness

Roger Hui

Papers

A generalization of the fast LUP matrix decomposition algorithm and applications