A large class of problems in symbolic computation can be expressed as the task of computing some polynomials; and arithmetic circuits form the most standard model for studying the complexity of such computations. This algebraic model of computation attracted a large amount of research in the last five decades, partially due to its simplicity and elegance. Being a more structured model than Boolean circuits, one could hope that the fundamental problems of theoretical computer science, such as separating P from NP, will be easier to solve for arithmetic circuits. However, in spite of the appearing simplicity and the vast amount of mathematical tools available, no major breakthrough has been seen. In fact, all the fundamental questions are still open for this model as well. Nevertheless, there has been a lot of progress in the area and beautiful results have been found, some in the last few years. As examples we mention the connection between polynomial identity testing and lower bounds of Kabanets and Impagliazzo, the lower bounds of Raz for multilinear formulas, and two new approaches for proving lower bounds: Geometric Complexity Theory and Elusive Functions.

The goal of this monograph is to survey the field of arithmetic circuit complexity, focusing mainly on what we find to be the most interesting and accessible research directions. We aim to cover the main results and techniques, with an emphasis on works from the last two decades. In particular, we discuss the recent lower bounds for multilinear circuits and formulas, the advances in the question of deterministically checking polynomial identities, and the results regarding reconstruction of arithmetic circuits. We do, however, also cover part of the classical works on arithmetic circuits. In order to keep this monograph at a reasonable length, we do not give full proofs of most theorems, but rather try to convey the main ideas behind each proof and demonstrate it, where possible, by proving some special cases.

https://www.cs.tau.ac.il/~shpilka/publications/SY10.pdf

Arithmetic Circuits: A Survey of Recent Results and Open Questions

In this work we consider representations of multivariate polynomials in F[x] of the form f(x) = Q1(x) +Q2(x) + . . .+Qs(x) , where the ei’s are positive integers and the Qi’s are arbitary multivariate polynomials of bounded degree. We give an explicit n-variate polynomial f of degree n such that any representation of the above form for f requires the number of summands s to be 2. Motivation. Let F be a field, F[x] be the set of n-variate polynomials over F and d ≥ 1 be an integer. For a polynomial f(x) ∈ F[x], we consider representations of the form f(x) = Q1 1 (x) +Q e2 2 (x) + . . .+Q es s (x), (1) where the Qi(x)’s are polynomials of degree at most d. We do this with an eye towards proving lower bounds for the number of summands s required to write some explicit polynomial f in the above form. Our motivation for this line of inquiry stems from some recent results and problems posed in the field of arithmetic complexity. Agrawal and Vinay [AV08] showed that proving exponential lower bounds for depth four arithmetic circuits implies exponential lower bounds for arbitrary depth arithmetic circuits. In our case, a representation of the form (1) above corresponds to computing f via a depth four ΣΠΣΠ arithmetic circuit where the bottommost layer of multiplication gates have fanin bounded by d and the second-last layer of multiplication gates actually consists of exponentiation gates of arbitrarily large degree (i.e. multiplication gates where all the incoming edges originate from a single node). Meanwhile Hrubes, Wigderson and Yehudayoff [HWY10] look at the situation where d = e1 = e2 = . . . = es = 2 and ask for a superlinear lower bound on the number of summands s for an explicit n-variate biquadratic polynomial f . They show that such a superlinear lower bound implies an exponential lower bound on the size of arithmetic circuits computing the noncommutative permanent. Finally Chen, Kayal and Wigderson [CKW11] pose the problem of proving lower bounds for bounded depth arithmetic circuits with addition and exponentiation gates. Our main theorem is a lower bound on the number of summands in any representation of the form (1) for an explicit polynomial. Theorem 1. (Lower bound for sum of powers). Let F be any field and F[x] be the ring of polynomials over the set of indeterminates x = (x1, x2, . . . , xn). Let e1, e2, . . . , es be positive integers and Q1, Q2, . . . , Qs ∈ F[x] be multivariate polynomials each of degree at most d. If Q1 1 +Q e2 2 + . . .+Q es s = (x1 · x2 · . . . · xn), then we must have that (log s) = Ω( n 2d ). In particular, if d is a constant then s = 2Ω(n). Remark 2. 1. The fact that the f in the lower bound above consists of a single monomial indicates above all the severe limitation of representations of the form (1). ∗Microsoft Research India, neeraka@microsoft.com 1 ISSN 1433-8092 Electronic Colloquium on Computational Complexity, Report No. 81 (2012) 2. An upper bound of 2n/d is an easy corollary of Fischer [Fis94]. Specifically, let F be an algebraically closed field with char(F) > n. Then for all integers d ≥ 1 there exist polynomials Q1, Q2, . . . , Qs each of degree d such that Q1 1 +Q e2 2 + . . .+Q es s = (x1 · x2 · . . . · xn), and the number of summands s is at most 2n/d. Fischer [Fis94] gives an explicit set of 2m−1 linear forms `1, `2, . . . , `2m such that (y1 · y2 · . . . · ym) = ∑

An exponential lower bound for the sum of powers of bounded degree polynomials.

We study the problem of obtaining efficient, deterministic, black-box polynomial identity testing algorithms (PIT) for algebraic branching programs (ABPs) that are read-once and oblivious. This class has an efficient, deterministic, white-box polynomial identity testing algorithm (due to Raz and Shpilka), but prior to this work there was no known such black-box algorithm. The main result of this work gives the first quasi-polynomial sized hitting sets for size S circuits from this class, when the order of the variables is known. As our hitting set is of size exp(lg2 S), this is analogous (in the terminology of boolean pseudorandom ness) to a seed-length of lg2 S, which is the seed length of the pseudorandom generators of Nisan and Impagliazzo-Nisan-Wigderson for read-once oblivious boolean branching programs. Thus our work can be seen as an algebraic analogue of these foundational results in boolean pseudorandom ness. Our results are stronger for branching programs of bounded width, where we give a hitting set of size exp(lg2 S/lglg S), corresponding to a seed length of lg2 S/lglg S. This is in stark contrast to the known results for read-once oblivious boolean branching programs of bounded width, where no pseudorandom generator (or hitting set) with seed length o(lg2 S) is known. Thus, while our work is in some sense an algebraic analogue of existing boolean results, the two regimes seem to have non-trivial differences. In follow up work, we strengthened a result of Mulmuley, and showed that derandomizing a particular case of the No ether Normalization Lemma is reducible to black-box PIT of read-once oblivious ABPs. Using the results of the present work, this gives a derandomization of No ether Normalization in that case, which Mulmuley conjectured would difficult due to its relations to problems in algebraic geometry. We also show that several other circuit classes can be black-box reduced to read-once oblivious ABPs, including set-multilinear ABPs (a generalization of depth-3 set-multilinear formulas), non-commutative ABPs (generalizing non-commutative formulas), and (semi-)diagonal depth-4 circuits (as introduced by Saxena). For set-multilinear ABPs and non-commutative ABPs, we give quasi-polynomial-time black-box PIT algorithms, where the latter case involves evaluations over the algebra of (D+1)x(D+1) matrices, where D is the depth of the ABP. For (semi-)diagonal depth-4 circuits, we obtain a black-box PIT algorithm (over any characteristic) whose run-time is quasi-polynomial in the runtime of Saxena's white-box algorithm, matching the concurrent work of Agrawal, Saha, and Saxena. Finally, by combining our results with the reconstruction algorithm of Klivans and Shpilka, we obtain deterministic reconstruction algorithms for the above circuit classes.

/pdf/quasipolynomial-time-identity-testing-of-non-commutative-and-97jt8ebjih.pdf

Quasipolynomial-Time Identity Testing of Non-commutative and Read-Once Oblivious Algebraic Branching Programs

In 1967, J. Edmonds introduced the problem of computing the rank over the rational function field of an $${n \times n}$$
 matrix T with integral homogeneous linear polynomials. In this paper, we consider the non-commutative version of Edmonds’ problem: compute the rank of T over the free skew field. This problem has been proposed, sometimes in disguise, from several different perspectives in the study of, for example, the free skew field itself (Cohn in J Symbol Log 38(2):309–314, 1973), matrix spaces of low rank (Fortin-Reutenauer in Semin Lothar Comb 52:B52f 2004), Edmonds’ original problem (Gurvits in J Comput Syst Sci 69(3):448–484, 2004), and more recently, non-commutative arithmetic circuits with divisions (Hrubes and Wigderson in Theory Comput 11:357-393, 2015. doi:
 10.4086/toc.2015.v011a014
 ). It is known that this problem relates to the following invariant ring, which we call the $${\mathbb{F}}$$
 -algebra of matrix semi-invariants, denoted as R(n, m). For a field $${\mathbb{F}}$$
 , it is the ring of invariant polynomials for the action of $${{\rm SL}(n, \mathbb{F}) \times
{\rm SL}(n, \mathbb{F})}$$
 on tuples of matrices—
 $${(A, C)\in {\rm SL}(n, \mathbb{F}) \times
{\rm SL}(n, \mathbb{F})}$$
 sends $${(B_{1}, \ldots, B_m)\in M(n, \mathbb{F})^{\oplus m}}$$
 to $${(AB_1 {C}^{{\rm T}}, \ldots, AB_m {C}^{\rm T})}$$
 . Then those T with non-commutative rank <  n correspond to those points in the nullcone of R(n, m). In particular, if the nullcone of R(n, m) is defined by elements of degree $${\leq \sigma}$$
 , then there follows a $${{\rm poly}(n,\sigma)}$$
 -time randomized algorithm to decide whether the non-commutative rank of T is full. To our knowledge, previously the best bound for $${\sigma}$$
 was $${O(n^{2}\cdot 4^{n^2})}$$
 over algebraically closed fields of characteristic 0 (Derksen in Proc Am Math Soc 129(4):955–964, 2001). We now state the main contributions of this paper:

Non-commutative Edmonds’ problem and matrix semi-invariants

Symbolic matrices in non-commuting variables, andthe related structural and algorithmic questions, have a remarkablenumber of diverse origins and motivations. They ariseindependently in (commutative) invariant theory and representationtheory, linear algebra, optimization, linear system theory,quantum information theory, and naturally in non-commutativealgebra.

A Deterministic Polynomial Time Algorithm for Non-commutative Rational Identity Testing

Monotone separations for constant degree polynomials

We investigate the arithmetic formula complexity of the elementary symmetric polynomials S(k,n). We show that every multilinear homogeneous formula computing S(k,n) has size at least k^(Omega(log k))n, and that product-depth d multilinear homogeneous formulas for S(k,n) have size at least 2^(Omega(k^{1/d}))n. Since S(n,2n) has a multilinear formula of size O(n^2), we obtain a superpolynomial separation between multilinear and multilinear homogeneous formulas. We also show that S(k,n) can be computed by homogeneous formulas of size k^(O(log k))n, answering a question of Nisan and Wigderson. Finally, we present a superpolynomial separation between monotone and non-monotone formulas in the noncommutative setting, answering a question of Nisan.

Homogeneous formulas and symmetric polynomials

We show that strong-enough lower bounds on monotone arithmetic circuits or the non-negative rank of a matrix imply unconditional lower bounds in arithmetic or Boolean circuit complexity. First, we show that if a polynomial f ∈ R[x1, . . . , xn] of degree d has an arithmetic circuit of size s then (x1 + · · · + xn + 1) + f has a monotone arithmetic circuit of size O(sd + n logn), for some > 0. Second, if f : {0, 1} → {0, 1} is a Boolean function, we associate with f an explicit exponentialsize matrix M(f) such that the Boolean circuit size of f is at least Ω(min >0(rk+(M(f) − J)) − 2n), where J is the all-ones matrix and rk+ denotes the non-negative rank of a matrix. In fact, the quantity min >0(rk+(M(f) − J)) characterizes how hard is it to distinguish rejecting and accepting inputs of f by means of a linear program. Finally, we introduce a proof system resembling the monotone calculus of Atserias et al. [2] and show that similar -sensitive lower bounds on monotone arithmetic circuits imply lower bounds on proof-size in the system.

On ε-sensitive monotone computations.

We initiate a direction for proving lower bounds on the size of non-commutative arithmetic circuits. This direction is based on a connection between lower bounds on the size of non-commutative arithmetic circuits and a problem about commutative degree four polynomials, the classical sum-of-squares problem: find the smallest n such that there exists an identity (x12+x22+•• + xk2)• (y1^2+y22+•• + yk2)= f12+f22+ ... +fn2, where each fi = fi(X,Y) is bilinear in X={x1,... ,xk} and Y={y1,..., yk}. Over the complex numbers, we show that a sufficiently strong super-linear lower bound on n in, namely, n ≥ k1+e with e >0, implies an exponential lower bound on the size of arithmetic circuits computing the non-commutative permanent.More generally, we consider such sum-of-squares identities for any M polynomial h(X,Y), namely: h(X,Y) = f12+f22+...+fn2.Again, proving n ≥ k1+e in for any explicit h over the complex numbers gives an exponential lower bound for the non-commutative permanent. Our proofs relies on several new structure theorems for non-commutative circuits, as well as a non-commutative analog of Valiant's completeness of the permanent.We proceed to prove such super-linear bounds in some restricted cases. We prove that n ≥ Ω(k6/5) in (1), if f1,..., fn are required to have integer coefficients. Over the real numbers, we construct an explicit M polynomial h such that n in (2) must be at least Ω(k2). Unfortunately, these results do not imply circuit lower bounds. We also present other structural results about non-commutative arithmetic circuits. We show that any non-commutative circuit computing an ordered non-commutative polynomial can be efficiently transformed to a syntactically multilinear circuit computing that polynomial. The permanent, for example, is ordered. Hence, lower bounds on the size of syntactically multilinear circuits computing the permanent imply unrestricted non-commutative lower bounds. We also prove an exponential lower bound on the size of non-commutative syntactically multilinear circuit computing an explicit polynomial. This polynomial is, however, not ordered and an unrestricted circuit lower bound does not follow.

Non-commutative circuits and the sum-of-squares problem.

We show that strong-enough lower bounds on monotone arithmetic circuits or the nonnegative rank of a matrix imply unconditional lower bounds in arithmetic or Boolean circuit complexity. First, we show that if a polynomial 
$$f\in \mathbb {R}[x_1,\dots , x_n]$$

 of degree d has an arithmetic circuit of size s then 
$$(x_1+\dots +x_n+1)^d+\epsilon f$$

 has a monotone arithmetic circuit of size 
$$O(sd^2+n\log n)$$

, for some 
$$\epsilon >0$$

. Second, if 
$$f:\{0,1\}^n\rightarrow \{0,1\}$$

 is a Boolean function, we associate with f an explicit exponential-size matrix M(f) such that the Boolean circuit size of f is at least 
$$\varOmega (\min _{\epsilon >0}(\mathrm{rk}_{+}(M(f)-\epsilon J))- 2n)$$

, where J is the all-ones matrix and 
$$\mathrm{rk}_{+}$$

 denotes the nonnegative rank of a matrix. In fact, the quantity 
$$\min _{\epsilon >0}(\mathrm{rk}_{+}(M(f)-\epsilon J))$$

 characterizes how hard is it to distinguish rejecting and accepting inputs of f by means of a linear program. Finally, we introduce a proof system resembling the monotone calculus of Atserias et al. (J Comput Syst Sci 65:626–638, 2002) and show that similar 
$$\epsilon $$

-sensitive lower bounds on monotone arithmetic circuits imply lower bounds on proof-size in the system.

Pavel Hrubeš

Papers

Monotone separations for constant degree polynomials

Homogeneous formulas and symmetric polynomials

On ε-sensitive monotone computations.

Non-commutative circuits and the sum-of-squares problem.

On $$\epsilon$$ ϵ -sensitive monotone computations