A large class of problems in symbolic computation can be expressed as the task of computing some polynomials; and arithmetic circuits form the most standard model for studying the complexity of such computations. This algebraic model of computation attracted a large amount of research in the last five decades, partially due to its simplicity and elegance. Being a more structured model than Boolean circuits, one could hope that the fundamental problems of theoretical computer science, such as separating P from NP, will be easier to solve for arithmetic circuits. However, in spite of the appearing simplicity and the vast amount of mathematical tools available, no major breakthrough has been seen. In fact, all the fundamental questions are still open for this model as well. Nevertheless, there has been a lot of progress in the area and beautiful results have been found, some in the last few years. As examples we mention the connection between polynomial identity testing and lower bounds of Kabanets and Impagliazzo, the lower bounds of Raz for multilinear formulas, and two new approaches for proving lower bounds: Geometric Complexity Theory and Elusive Functions.

The goal of this monograph is to survey the field of arithmetic circuit complexity, focusing mainly on what we find to be the most interesting and accessible research directions. We aim to cover the main results and techniques, with an emphasis on works from the last two decades. In particular, we discuss the recent lower bounds for multilinear circuits and formulas, the advances in the question of deterministically checking polynomial identities, and the results regarding reconstruction of arithmetic circuits. We do, however, also cover part of the classical works on arithmetic circuits. In order to keep this monograph at a reasonable length, we do not give full proofs of most theorems, but rather try to convey the main ideas behind each proof and demonstrate it, where possible, by proving some special cases.

https://www.cs.tau.ac.il/~shpilka/publications/SY10.pdf

Arithmetic Circuits: A Survey of Recent Results and Open Questions

In this work we consider representations of multivariate polynomials in F[x] of the form f(x) = Q1(x) +Q2(x) + . . .+Qs(x) , where the ei’s are positive integers and the Qi’s are arbitary multivariate polynomials of bounded degree. We give an explicit n-variate polynomial f of degree n such that any representation of the above form for f requires the number of summands s to be 2. Motivation. Let F be a field, F[x] be the set of n-variate polynomials over F and d ≥ 1 be an integer. For a polynomial f(x) ∈ F[x], we consider representations of the form f(x) = Q1 1 (x) +Q e2 2 (x) + . . .+Q es s (x), (1) where the Qi(x)’s are polynomials of degree at most d. We do this with an eye towards proving lower bounds for the number of summands s required to write some explicit polynomial f in the above form. Our motivation for this line of inquiry stems from some recent results and problems posed in the field of arithmetic complexity. Agrawal and Vinay [AV08] showed that proving exponential lower bounds for depth four arithmetic circuits implies exponential lower bounds for arbitrary depth arithmetic circuits. In our case, a representation of the form (1) above corresponds to computing f via a depth four ΣΠΣΠ arithmetic circuit where the bottommost layer of multiplication gates have fanin bounded by d and the second-last layer of multiplication gates actually consists of exponentiation gates of arbitrarily large degree (i.e. multiplication gates where all the incoming edges originate from a single node). Meanwhile Hrubes, Wigderson and Yehudayoff [HWY10] look at the situation where d = e1 = e2 = . . . = es = 2 and ask for a superlinear lower bound on the number of summands s for an explicit n-variate biquadratic polynomial f . They show that such a superlinear lower bound implies an exponential lower bound on the size of arithmetic circuits computing the noncommutative permanent. Finally Chen, Kayal and Wigderson [CKW11] pose the problem of proving lower bounds for bounded depth arithmetic circuits with addition and exponentiation gates. Our main theorem is a lower bound on the number of summands in any representation of the form (1) for an explicit polynomial. Theorem 1. (Lower bound for sum of powers). Let F be any field and F[x] be the ring of polynomials over the set of indeterminates x = (x1, x2, . . . , xn). Let e1, e2, . . . , es be positive integers and Q1, Q2, . . . , Qs ∈ F[x] be multivariate polynomials each of degree at most d. If Q1 1 +Q e2 2 + . . .+Q es s = (x1 · x2 · . . . · xn), then we must have that (log s) = Ω( n 2d ). In particular, if d is a constant then s = 2Ω(n). Remark 2. 1. The fact that the f in the lower bound above consists of a single monomial indicates above all the severe limitation of representations of the form (1). ∗Microsoft Research India, neeraka@microsoft.com 1 ISSN 1433-8092 Electronic Colloquium on Computational Complexity, Report No. 81 (2012) 2. An upper bound of 2n/d is an easy corollary of Fischer [Fis94]. Specifically, let F be an algebraically closed field with char(F) > n. Then for all integers d ≥ 1 there exist polynomials Q1, Q2, . . . , Qs each of degree d such that Q1 1 +Q e2 2 + . . .+Q es s = (x1 · x2 · . . . · xn), and the number of summands s is at most 2n/d. Fischer [Fis94] gives an explicit set of 2m−1 linear forms `1, `2, . . . , `2m such that (y1 · y2 · . . . · ym) = ∑

An exponential lower bound for the sum of powers of bounded degree polynomials.

We study the problem of obtaining efficient, deterministic, black-box polynomial identity testing algorithms (PIT) for algebraic branching programs (ABPs) that are read-once and oblivious. This class has an efficient, deterministic, white-box polynomial identity testing algorithm (due to Raz and Shpilka), but prior to this work there was no known such black-box algorithm. The main result of this work gives the first quasi-polynomial sized hitting sets for size S circuits from this class, when the order of the variables is known. As our hitting set is of size exp(lg2 S), this is analogous (in the terminology of boolean pseudorandom ness) to a seed-length of lg2 S, which is the seed length of the pseudorandom generators of Nisan and Impagliazzo-Nisan-Wigderson for read-once oblivious boolean branching programs. Thus our work can be seen as an algebraic analogue of these foundational results in boolean pseudorandom ness. Our results are stronger for branching programs of bounded width, where we give a hitting set of size exp(lg2 S/lglg S), corresponding to a seed length of lg2 S/lglg S. This is in stark contrast to the known results for read-once oblivious boolean branching programs of bounded width, where no pseudorandom generator (or hitting set) with seed length o(lg2 S) is known. Thus, while our work is in some sense an algebraic analogue of existing boolean results, the two regimes seem to have non-trivial differences. In follow up work, we strengthened a result of Mulmuley, and showed that derandomizing a particular case of the No ether Normalization Lemma is reducible to black-box PIT of read-once oblivious ABPs. Using the results of the present work, this gives a derandomization of No ether Normalization in that case, which Mulmuley conjectured would difficult due to its relations to problems in algebraic geometry. We also show that several other circuit classes can be black-box reduced to read-once oblivious ABPs, including set-multilinear ABPs (a generalization of depth-3 set-multilinear formulas), non-commutative ABPs (generalizing non-commutative formulas), and (semi-)diagonal depth-4 circuits (as introduced by Saxena). For set-multilinear ABPs and non-commutative ABPs, we give quasi-polynomial-time black-box PIT algorithms, where the latter case involves evaluations over the algebra of (D+1)x(D+1) matrices, where D is the depth of the ABP. For (semi-)diagonal depth-4 circuits, we obtain a black-box PIT algorithm (over any characteristic) whose run-time is quasi-polynomial in the runtime of Saxena's white-box algorithm, matching the concurrent work of Agrawal, Saha, and Saxena. Finally, by combining our results with the reconstruction algorithm of Klivans and Shpilka, we obtain deterministic reconstruction algorithms for the above circuit classes.

/pdf/quasipolynomial-time-identity-testing-of-non-commutative-and-97jt8ebjih.pdf

Quasipolynomial-Time Identity Testing of Non-commutative and Read-Once Oblivious Algebraic Branching Programs

In 1967, J. Edmonds introduced the problem of computing the rank over the rational function field of an $${n \times n}$$
 matrix T with integral homogeneous linear polynomials. In this paper, we consider the non-commutative version of Edmonds’ problem: compute the rank of T over the free skew field. This problem has been proposed, sometimes in disguise, from several different perspectives in the study of, for example, the free skew field itself (Cohn in J Symbol Log 38(2):309–314, 1973), matrix spaces of low rank (Fortin-Reutenauer in Semin Lothar Comb 52:B52f 2004), Edmonds’ original problem (Gurvits in J Comput Syst Sci 69(3):448–484, 2004), and more recently, non-commutative arithmetic circuits with divisions (Hrubes and Wigderson in Theory Comput 11:357-393, 2015. doi:
 10.4086/toc.2015.v011a014
 ). It is known that this problem relates to the following invariant ring, which we call the $${\mathbb{F}}$$
 -algebra of matrix semi-invariants, denoted as R(n, m). For a field $${\mathbb{F}}$$
 , it is the ring of invariant polynomials for the action of $${{\rm SL}(n, \mathbb{F}) \times
{\rm SL}(n, \mathbb{F})}$$
 on tuples of matrices—
 $${(A, C)\in {\rm SL}(n, \mathbb{F}) \times
{\rm SL}(n, \mathbb{F})}$$
 sends $${(B_{1}, \ldots, B_m)\in M(n, \mathbb{F})^{\oplus m}}$$
 to $${(AB_1 {C}^{{\rm T}}, \ldots, AB_m {C}^{\rm T})}$$
 . Then those T with non-commutative rank <  n correspond to those points in the nullcone of R(n, m). In particular, if the nullcone of R(n, m) is defined by elements of degree $${\leq \sigma}$$
 , then there follows a $${{\rm poly}(n,\sigma)}$$
 -time randomized algorithm to decide whether the non-commutative rank of T is full. To our knowledge, previously the best bound for $${\sigma}$$
 was $${O(n^{2}\cdot 4^{n^2})}$$
 over algebraically closed fields of characteristic 0 (Derksen in Proc Am Math Soc 129(4):955–964, 2001). We now state the main contributions of this paper:

Non-commutative Edmonds’ problem and matrix semi-invariants

Symbolic matrices in non-commuting variables, andthe related structural and algorithmic questions, have a remarkablenumber of diverse origins and motivations. They ariseindependently in (commutative) invariant theory and representationtheory, linear algebra, optimization, linear system theory,quantum information theory, and naturally in non-commutativealgebra.

A Deterministic Polynomial Time Algorithm for Non-commutative Rational Identity Testing

We investigate the arithmetic formula complexity of the elementary symmetric polynomials $${S^k_n}$$. We show that every multilinear homogeneous formula computing $${S^k_n}$$ has size at least $${k^{\Omega(\log k)}n}$$, and that product-depth d multilinear homogeneous formulas for $${S^k_n}$$ have size at least $${2^{\Omega(k^{1/d})}n}$$. Since $${S^{n}_{2n}}$$ has a multilinear formula of size O(n 2), we obtain a superpolynomial separation between multilinear and multilinear homogeneous formulas. We also show that $${S^k_n}$$ can be computed by homogeneous formulas of size $${k^{O(\log k)}n}$$, answering a question of Nisan and Wigderson. Finally, we present a superpolynomial separation between monotone and non-monotone formulas in the noncommutative setting, answering a question of Nisan.

https://www.math.ias.edu/files/Hrubes_Symmetric2.pdf

Homogeneous Formulas and Symmetric Polynomials

We initiate a direction for proving lower bounds on the size of non-commutative arithmetic circuits. This direction is based on a connection between lower bounds on the size of non-commutative arithmetic circuits and a problem about commutative degree four polynomials, the classical sum-of-squares problem: find the smallest n such that there exists an identity (x12+x22+•• + xk2)• (y1^2+y22+•• + yk2)= f12+f22+ ... +fn2, where each fi = fi(X,Y) is bilinear in X={x1,... ,xk} and Y={y1,..., yk}. Over the complex numbers, we show that a sufficiently strong super-linear lower bound on n in, namely, n ≥ k1+e with e >0, implies an exponential lower bound on the size of arithmetic circuits computing the non-commutative permanent.More generally, we consider such sum-of-squares identities for any M polynomial h(X,Y), namely: h(X,Y) = f12+f22+...+fn2.Again, proving n ≥ k1+e in for any explicit h over the complex numbers gives an exponential lower bound for the non-commutative permanent. Our proofs relies on several new structure theorems for non-commutative circuits, as well as a non-commutative analog of Valiant's completeness of the permanent.We proceed to prove such super-linear bounds in some restricted cases. We prove that n ≥ Ω(k6/5) in (1), if f1,..., fn are required to have integer coefficients. Over the real numbers, we construct an explicit M polynomial h such that n in (2) must be at least Ω(k2). Unfortunately, these results do not imply circuit lower bounds. We also present other structural results about non-commutative arithmetic circuits. We show that any non-commutative circuit computing an ordered non-commutative polynomial can be efficiently transformed to a syntactically multilinear circuit computing that polynomial. The permanent, for example, is ordered. Hence, lower bounds on the size of syntactically multilinear circuits computing the permanent imply unrestricted non-commutative lower bounds. We also prove an exponential lower bound on the size of non-commutative syntactically multilinear circuit computing an explicit polynomial. This polynomial is, however, not ordered and an unrestricted circuit lower bound does not follow.

/pdf/non-commutative-circuits-and-the-sum-of-squares-problem-2gawvz1c2r.pdf

Non-commutative circuits and the sum-of-squares problem

We initiate the study of the complexity of arithmetic circuits with division gates over non-commuting variables. Such circuits and formulas compute non-commutative rational functions, which, despite their name, can no longer be expressed as ratios of polynomials. We prove some lower and upper bounds, completeness and simulation results, as follows. If X is n x n matrix consisting of n2 distinct mutually non-commuting variables, we show that: (i). X-1 can be computed by a circuit of polynomial size, (ii). every formula computing some entry of X-1 must have size at least 2Ω(n). We also show that matrix inverse is complete in the following sense: (i). Assume that a non-commutative rational function f can be computed by a formula of size s. Then there exists an invertible 2s x 2s-matrix A whose entries are variables or field elements such that f is an entry of A-1. (ii). If f is a non-commutative polynomial computed by a formula without inverse gates then A can be taken as an upper triangular matrix with field elements on the diagonal. We show how divisions can be eliminated from non-commutative circuits and formulae which compute polynomials, and we address the non-commutative version of the "rational function identity testing" problem. As it happens, the complexity of both of these procedures depends on a single open problem in invariant theory.

/pdf/non-commutative-arithmetic-circuits-with-division-4b8swmkjjk.pdf

Non-commutative arithmetic circuits with division

/pdf/non-commutative-circuits-and-the-sum-of-squares-problem-1zeh2cxa3f.pdf

/pdf/non-commutative-arithmetic-circuits-with-division-498hcgcore.pdf

Pavel Hrubeš

Papers

Homogeneous Formulas and Symmetric Polynomials

Non-commutative circuits and the sum-of-squares problem

Non-commutative arithmetic circuits with division

Non-commutative circuits and the sum-of-squares problem

Non-Commutative Arithmetic Circuits with Division