We consider a notion of probabilistic rank and probabilistic sign-rank of a matrix, which measure the extent to which a matrix can be probabilistically represented by low-rank matrices. We demonstrate several connections with matrix rigidity, communication complexity, and circuit lower bounds. The most interesting outcomes are: The Walsh-Hadamard Transform is Not Very Rigid. We give surprising upper bounds on the rigidity of a family of matrices whose rigidity has been extensively studied, and was conjectured to be highly rigid. For the 2n X 2n Walsh-Hadamard transform Hn (a.k.a. Sylvester matrices, a.k.a. the communication matrix of Inner Product modulo 2), we show how to modify only 2e n entries in each row and make the rank of Hn drop below 2n(1-Ω(e2/log(1/ee))), for all small e > 0, over any field. That is, it is not possible to prove arithmetic circuit lower bounds on Hadamard matrices such as Hn, via L. Valiant's matrix rigidity approach. We also show non-trivial rigidity upper bounds for Hn with smaller target rank. Matrix Rigidity and Threshold Circuit Lower Bounds. We give new consequences of rigid matrices for Boolean circuit complexity. First, we show that explicit n X n Boolean matrices which maintain rank at least 2(logn)1-δ after n2/2(logn)δ/2 modified entries (over any field, for any δ > 0) would yield an explicit function that does not have sub-quadratic-size AC0 circuits with two layers of arbitrary linear threshold gates. Second, we prove that explicit 0/1 matrices over ℝ which are modestly more rigid than the best known rigidity lower bounds for sign-rank would imply exponential-gate lower bounds for the infamously difficult class of depth-two linear threshold circuits with arbitrary weights on both layers. In particular, we show that matrices defined by these seemingly-difficult circuit classes actually have low probabilistic rank and sign-rank, respectively. An Equivalence Between Communication, Probabilistic Rank, and Rigidity. It has been known since Razborov [1989] that explicit rigidity lower bounds would resolve longstanding lower-bound problems in communication complexity, but it seemed possible that communication lower bounds could be proved without making progress on matrix rigidity. We show that for every function f which is randomly self-reducible in a natural way (the inner product mod 2 is an example), bounding the communication complexity of f (in a precise technical sense) is equivalent to bounding the rigidity of the matrix of f, via an equivalence with probabilistic rank.

https://dl.acm.org/doi/pdf/10.1145/3055399.3055484

Probabilistic rank and matrix rigidity

A de Morgan formula over Boolean variables x1,…,xn is a binary tree whose internal nodes are marked with AND or OR gates and whose leaves are marked with variables or their negation. We define the size of the formula as the number of leaves in it. Proving that some explicit function (in P or NP) requires a large formula is a central open question in computational complexity. While we believe that some explicit functions require exponential formula size, currently the best lower bound for an explicit function is the Ω(n3) lower bound for Andreev's function. A long line of work in quantum query complexity, culminating in the work of Reichardt [SODA, 2011], proved that for any formula of size s, there exists a polynomial of degree at most O(√s) that approximates the formula up to a small point-wise error. This is a classical theorem, arguing about polynomials and formulae, however the only known proof for it involves quantum algorithms. We apply Reichardt result to obtain the following: (1) We show how to trade average-case hardness in exchange for size. More precisely, we show that if a function f cannot be computed correctly on more than 1/2 + 2-k of the inputs by any formula of size at most s, then computing f exactly requires formula size at least Ω(k) · s. As an application, we improve the state of the art formula size lower bounds for explicit functions by a factor of Ω(logn). (2) We prove that the bipartite formula size of the Inner-Product function is Ω(n2). (A bipartite formula on Boolean variables x1,…,xn and y1, …, yn is a binary tree whose internal nodes are marked with AND or OR gates and whose leaves can compute any function of either the x or y variables.) We show that any bipartite formula for the Inner-Product modulo 2 function, namely IP(x,y) = Σi=1n xi yi (mod 2), must be of size Ω(n2), which is tight up to logarithmic factors. To the best of our knowledge, this is the first super-linear lower bound on the bipartite formula complexity of any explicit function.

https://dl.acm.org/doi/pdf/10.1145/3055399.3055472

Formula lower bounds via the quantum method

The best known lower bounds for the circuit class TC0 are only slightly super-linear. Similarly, the best known algorithm for derandomization of this class is an algorithm for quantified derandomization (i.e., a weak type of derandomization) of circuits of slightly super-linear size. In this paper we show that even very mild quantitative improvements of either of the two foregoing results would already imply super-polynomial lower bounds for TC0. Specifically: (1) If for every c>1 and sufficiently large d∈ℕ it holds that n-bit TC0 circuits of depth d require n1+c−d wires to compute certain NC1-complete functions, then TC0≠NC1. In fact, even lower bounds for TC0 circuits of size n1+c−d against these functions when c>1 is fixed and sufficiently small would yield lower bounds for polynomial-sized circuits. Lower bounds of the form n1+c−d against these functions are already known, but for a fixed c≈2.41 that is too large to yield new lower bounds via our results. (2) If there exists a deterministic algorithm that gets as input an n-bit TC0 circuit of depth d and n1+(1.61)−d wires, runs in time 2no(1), and distinguishes circuits that accept at most B(n)=2n1−(1.61)−d inputs from circuits that reject at most B(n) inputs, then NEXP⊈TC0. An algorithm for this “quantified derandomization” task is already known, but it works only when the number of wires is n1+c−d, for c>30, and with a smaller B(n)≈2n1−(30/c)d. Intuitively, the “takeaway” message from our work is that the gap between currently-known results and results that would suffice to get super-polynomial lower bounds for TC0 boils down to the precise constant c>1 in the bound n1+c−d on the number of wires. Our results improve previous results of Allender and Koucký (2010) and of the second author (2018), respectively, whose hypotheses referred to circuits with n1+c/d wires (rather than n1+c−d wires). We also prove results similar to two results above for other circuit classes (i.e., ACC0 and CC0).

Bootstrapping results for threshold circuits “just beyond” known lower bounds

With the advent of large-scale neuromorphic platforms, we seek to better understand the applications of neuromorphic computing to more general-purpose computing domains. Graph analysis problems have grown increasingly relevant in the wake of readily available massive data. We demonstrate that a broad class of combinatorial and graph problems known as dynamic programs enjoy simple and efficient neuromorphic implementations, by developing a general technique to convert dynamic programs to spiking neuromorphic algorithms. Dynamic programs have been studied for over 50 years and have dozens of applications across many fields.

Dynamic Programming with Spiking Neural Computing

We prove an average-case depth hierarchy theorem for Boolean circuits over the standard basis of AND, OR, and NOT gates. Our hierarchy theorem says that for every d ≥ 2, there is an explicit n-variable Boolean function f, computed by a linear-size depth-d formula, which is such that any depth-(d−1) circuit that agrees with f on (1/2 + on(1)) fraction of all inputs must have size exp(nΩ (1/d)). This answers an open question posed by Hastad in his Ph.D. thesis (Hastad 1986b).Our average-case depth hierarchy theorem implies that the polynomial hierarchy is infinite relative to a random oracle with probability 1, confirming a conjecture of Hastad (1986a), Cai (1986), and Babai (1987). We also use our result to show that there is no “approximate converse” to the results of Linial, Mansour, Nisan (Linial et al. 1993) and (Boppana 1997) on the total influence of bounded-depth circuits.A key ingredient in our proof is a notion of random projections which generalize random restrictions.

https://dl.acm.org/doi/pdf/10.1145/3095799

PudlákPavel

Papers

Threshold circuits of bounded depth