Showing papers in "Electronic Colloquium on Computational Complexity in 1996"

Networks of Spiking Neurons: The Third Generation of Neural Network Models

Abstract: -The computational power of formal models for networks of spiking neurons is compared with that of other neural network models based on McCulloch Pitts neurons (i.e., threshold gates), respectively, sigmoidal gates. In particular it is shown that networks of spiking neurons are, with regard to the number of neurons that are needed, computationally more powerful than these other neural network models. A concrete biologically relevant function is exhibited which can be computed by a single spiking neuron (for biologically reasonable values o f its parameters), but which requires hundreds of hidden units on a sigmoidal neural net. On the other hand, it is known that any function that can be computed by a small sigmoidal neural net can also be computed by a small network of spiking neurons. This article does not assume prior knowledge about spiking neurons, and it contains an extensive list o f references to the currently available literature on computations in networks of spiking neurons and relevant results from neurobiology. © 1997 Elsevier Science Ltd. All rights reserved. Keywords--Spiking neuron, Integrate-and-fire neutron, Computational complexity, Sigmoidal neural nets, Lower bounds. 1. D E F I N I T I O N S AND M O T I V A T I O N S If one classifies neural network models according to their computational units, one can distinguish three different generations. The f irst generation is based on M c C u l l o c h P i t t s neurons as computational units. These are also referred to as perceptrons or threshold gates. They give rise to a variety of neural network models such as multilayer perceptrons (also called threshold circuits), Hopfield nets, and Boltzmann machines. A characteristic feature of these models is that they can only give digital output. In fact they are universal for computations with digital input and output, and every boolean function can be computed by some multilayer perceptron with a single hidden layer. The second generation is based on computational units that apply an "activation function" with a continuous set of possible output values to a weighted sum (or polynomial) of the inputs. Common activation functions are the s igmoid func t ion a(y) = 1/(1 + e -y) and the linear Acknowledgements: I would like to thank Eduardo Sontag and an anonymous referee for their helpful comments. Written under partial support by the Austrian Science Fund. Requests for reprints should be sent to W. Maass, Institute for Theoretical Computer Science, Technische Universit~it Graz, Klosterwiesgasse 32/2, A-8010, Graz, Austria; tel. +43 316 873-5822; fax: +43 316 873-5805; e-mail: maass@igi,tu-graz.ac.at saturated function 7r with 7r(y) = y for 0 --< y --< 1, 7r(y) = 0 for y < 0, lr(y) = 1 for y > 1. Besides piecewise polynomial activation functions we consider in this paper also "piecewise exponential" activation functions, whose pieces can be defined by expressions involving exponentiation (such as the definition of a). Typical examples for networks from this second generation are feedforward and recurrent sigmoidal neural nets, as well as networks of radial basis function units. These nets are also able to compute (with the help of thresholding at the network output) arbitrary boolean functions. Actually it has been shown that neural nets from the second generation can compute certain boolean functions with f e w e r gates than neural nets from the first generation (Maass, Schnitger, & Sontag, 1991; DasGupta & Schnitger, 1993). In addition, neural nets from the second generation are able to compute functions with analog input and output. In fact they are universal for analog computations in the sense that any continuous function with a compact domain and range can be approximated arbitrarily well (with regard to uniform convergence, i.e., the L= norm) by a network of this type with a single hidden layer. Another characteristic feature of this second generation of neural network models is that they support learning algorithms that are based on gradient descent such as backprop.

Property Testing and its connection to Learning and Approximation

Abstract: In this paper, we consider the question of determining whether a function f has property P or is e-far from any function with property P. A property testing algorithm is given a sample of the value of f on instances drawn according to some distribution. In some cases, it is also allowed to query f on instances of its choice. We study this question for different properties and establish some connections to problems in learning theory and approximation.In particular, we focus our attention on testing graph properties. Given access to a graph G in the form of being able to query whether an edge exists or not between a pair of vertices, we devise algorithms to test whether the underlying graph has properties such as being bipartite, k-Colorable, or having a p-Clique (clique of density p with respect to the vertex set). Our graph property testing algorithms are probabilistic and make assertions that are correct with high probability, while making a number of queries that is independent of the size of the graph. Moreover, the property testing algorithms can be used to efficiently (i.e., in time linear in the number of vertices) construct partitions of the graph that correspond to the property being tested, if it holds for the input graph.

Generating Hard Instances of Lattice Problems

Abstract: We give a random class of lattices in Z n so that, if there is a probabilistic polynomial time algorithm which nds a short vector in a random lattice with a probability of at least 1 2 then there is also a probabilistic polynomial time algorithm which solves the following three lattice problems in every lattice in Z n with a probability exponentially close to one. (1) Find the length of a shortest nonzero vector in an n-dimensional lattice, approximately, up to a polynomial factor. (2) Find the shortest nonzero vector in an n-dimensional lattice L where the shortest vector v is unique in the sense that any other vector whose length is at most n c kvk is parallel to v, where c is a suuciently large absolute constant. (3) Find a basis b 1 ; :::; b n in the n-dimensional lattice L whose length, deened as max n i=1 kb i k, is the smallest possible up to a polynomial factor. A large number of the existing techniques of cryptography include the generation of a speciic instance of a problem in NP (together with a solution) which for some reason is thought to be diicult to solve. As an example we may think about factor-ization. Here a party of a cryptographic protocol is supposed to provide a composite number m so that the factorization of m is known to her but she has some serious reason to believe that nobody else will be able to factor m. The most compelling reason for such a belief would be a mathematical proof of the fact that the prime factors of m cannot be found in less then k step in some realistic model of computation, where k is a very large number. For the moment we do not have any proof of this type, neither for speciic numerical values of m and k, nor in some assymptotic sense. In spite of the lack of mathematical proofs, in two cases at least, we may expect that a problem will be diicult to solve. One is the class of NP-complete problems. Here we may say that if there is a problem at all which is diicult to solve, then an NP-complete problem will provide such an example. The other case is, if the problem is a very famous question (e.g. prime factorization), which for a long time were unsuccesfully attacked by …

Quantum measurements and the Abelian Stabilizer Problem

Abstract: We present a polynomial quantum algorithm for the Abelian stabilizer problem which includes both factoring and the discrete logarithm. Thus we extend famous Shor’s results [7]. Our method is based on a procedure for measuring an eigenvalue of a unitary operator. Another application of this procedure is a polynomial quantum Fourier transform algorithm for an arbitrary finite Abelian group. The paper also contains a rather detailed introduction to the theory of quantum computation.

Public-Key Cryptosystems from Lattice Reduction Problems

Abstract: We present a new proposal for a trapdoor one-way function, from which we derive public-key encryption and digital signatures The security of the new construction is based on the conjectured computational difficulty of lattice-reduction problems, providing a possible alternative to existing public-key encryption algorithms and digital signatures such as RSA and DSS

On the Construction of Pseudo-Random Permutations: Luby-Rackoff Revisited

Abstract: Luby and Rackoff showed a method for constructing a pseudo-random permutation from a pseudo-random function. The method is based on composing four (or three for weakened security) so called Feistel permutations each of which requires the evaluation of a pseudo-random function. We reduce somewhat the complexity of the construction and simplify its proof of security by showing that two Feistel permutations are sufficient together with initial and final pair-wise independent permutations. The revised construction and proof provide a framework in which similar constructions may be brought up and their security can be easily proved. We demonstrate this by presenting some additional adjustments of the construction that achieve the following: Reduce the success probability of the adversary. Provide a construction of pseudo-random permutations with large input size using pseudo-random functions with small input size. Provide a construction of a pseudo-random permutation using a single pseudo-random function.

A Public-Key Cryptosystem with Worst-Case/Average-Case Equivalence

Abstract: We present a probabilistic public key cryptosystem which is secure unless the worst case of the following lattice problem can be solved in polynomial time: “Find the shortest nonzero vector in an n dimensional lattice L where the shortest vector v is unique in the sense that any other vector whose length is at most n’ [lull is parallel to v.”

Collision-Free Hashing from Lattice Problems

Abstract: In 1995, Ajtai described a construction of one-way functions whose security is equivalent to the difficulty of some well known approximation problems in lattices. We show that essentially the same construction can also be used to obtain collision-free hashing. This paper contains a self-contained proof sketch of Ajtai's result.

Optimal Search in Trees

Abstract: It is well known that the optimal solution for searching in a finite total order set is the binary search. In binary search we divide the set into two {open_quotes}halves{close_quotes}, by querying the middle element, and continue the search on the suitable half. What is the equivalent of binary search, when the set P is partially ordered? A query in this case is to a point x {element_of} P, with two possible answers:{open_quote}yes{close_quote}, indicates that the required element is {open_quotes}below{close_quotes} x, or {open_quote}no{close_quote} if the element is not bellow x. We show that the problem of computing an optimal strategy for search in Posets that are tree-like (or forests) is polynomial in the size of the tree, and requires at most O(n{sup 2} log{sup 2} n) steps. Optimal solutions of such search problems are often needed in program testing and debugging, where a given program is represented as a tree and a bug should be found using a minimal set of queries.

The Boolean Isomorphism Problem

Abstract: We investigate the computational complexity of the Boolean isomorphism problem (BI): on input of two Boolean formulas F and G decide whether there exists a permutation of the variables of G such that F and G become equivalent. Our main result is a one-round interactive proof for BI, where the verifier has access to an NP oracle. To obtain this, we use a recent result from learning theory by N. Bshouty et al. (1995), that Boolean formulas can be learned probabilistically with equivalence queries and access to an NP oracle. As a consequence, BI cannot be /spl Sigma//sub 2//sup p/ complete unless the polynomial hierarchy collapses. This solves an open problem posed previously. Further properties of BI are shown: BI has And- and Or-functions, the counting version, BI, can be computed in polynomial time relative to BI, and BI is self-reducible.

A large lower bound for 1-branching programs

Abstract: Branching programs (b. p.'s) or decision diagrams are a general graph-based model of sequential computation. B.p.'s of polynomial size are a nonuniform counterpart of LOG. Lower bounds for diierent kinds of restricted b. p.'s are intensively investigated. An important restriction are so called 1{b. p.'s, where each computation reads each input bit at most once. There is a series of lower bounds for 1{b. p.'s. The largest known lower bound was 2 n=2000 for a function of n variables, see 11]. In the present paper, a lower bound of 2 n?o(n) is given.

Optimal attribute-efficient learning of disjunction, parity, and threshold functions

Abstract: Decision trees are a very general computation model. Here the problem is to identify a Boolean function f out of a given set of Boolean functions F by asking for the value of f at adaptively chosen inputs. For classes F consisting of functions which may be obtained from one function g on n inputs by replacing arbitrary n−k inputs by given constants this problem is known as attribute-efficient learning with k essential attributes. Results on general classes of functions are known. More precise and often optimal results are presented for the cases where g is one of the functions disjunction, parity or threshold.

Succinct Circuit Representations and Leaf Language Classes are Basically the same Concept

Abstract: This note connects two topics of complexity theory: The topic of succinct circuit representations initiated by Galperin and Wigderson (1983), and the topic of leaf language classes initiated by Bovet et al. (1992). It will be shown for any language that its succinct version is polynomial-time many-one complete for the leaf language class determined by it.

Visual Cryptography for General Access Structures

Abstract: A visual cryptography scheme for a set P ofnparticipants is a method of encoding a secret imageSIintonshadow images called shares, where each participant in P receives one share. Certain qualified subsets of participants can “visually” recover the secret image, but other, forbidden, sets of participants have no information (in an information-theoretic sense) onSI. A “visual” recovery for a setX?P consists of xeroxing the shares given to the participants inXonto transparencies, and then stacking them. The participants in a qualified setXwill be able to see the secret image without any knowledge of cryptography and without performing any cryptographic computation. In this paper we propose two techniques for constructing visual cryptography schemes for general access structures. We analyze the structure of visual cryptography schemes and we prove bounds on the size of the shares distributed to the participants in the scheme. We provide a novel technique for realizingkout ofnthreshold visual cryptography schemes. Our construction forkout ofnvisual cryptography schemes is better with respect to pixel expansion than the one proposed by M. Naor and A. Shamir (Visual cryptography,in“Advances in Cryptology?Eurocrypt '94” CA. De Santis, Ed.), Lecture Notes in Computer Science, Vol. 950, pp. 1?12, Springer-Verlag, Berlin, 1995) and for the case of 2 out ofnis the best possible. Finally, we consider graph-based access structures, i.e., access structures in which any qualified set of participants contains at least an edge of a given graph whose vertices represent the participants of the scheme.

Sparse Hard Sets for P Yields Space-Efficient Algorithms

Abstract: In 1978, Hartmanis conjectured that there exist no sparse complete sets for P under logspace many-one reductions. In this paper, in support of the conjecture, it is shown that if P has sparse hard sets under logspace many-one reductions, then P is a subset of DSPACE[log^2 n].

On Learning Branching Programs and Small Depth Circuits

Abstract: We study the learnability of branching programs and small-depth circuits with modular and threshold gates in both the exact and PAC learning models with and without membership queries. Our results extend earlier works [11, 18, 15] and exhibit further applications of multiplicity automata [7] in learning theory.

Structure in Approximation Classes

Abstract: The study of the approximability properties of NP-hard optimization problems has recently made great advances mainly due to the results obtained in the field of proof checking. The last important breakthrough proves the APX-completeness of several important optimization problems and thus reconciles "two distinct views of approximation classes: syntactic and computational" [S. Khanna et al., in Proc. 35th IEEE Symp. on Foundations of Computer Science, IEEE Computer Society Press, Los Alamitos, CA, 1994, pp. 819--830]. In this paper we obtain new results on the structure of several computationally-defined approximation classes. In particular, after defining a new approximation preserving reducibility to be used for as many approximation classes as possible, we give the first examples of natural NPO-complete problems and the first examples of natural APX-intermediate problems. Moreover, we state new connections between the approximability properties and the query complexity of NPO problems.

Learning Multivariate Polynomials from Substitution and Equivalence Queries

Abstract: It has been shown in previous recent work that multiplicity au-tomata are predictable from multiplicity and equivalence queries. In this paper we generalize related notions in a matrix representation and obtain a basis for the solution of a number of open problems in learn-ability theory. Membership queries are generalized to \\substitution\" queries for learning non-boolean functions and provide the value of the target function for a given input. In particular, using substitution and equivalence queries, we prove the learnability of the boolean XOR of terms, XOR decision trees, decision trees with integer variables and less than conditions, multivariate polynomials over a nite eld and rational functions over a xed nite eld. We also provide results for the case of innnite or large elds.

A Compendium of Problems Complete for Symmetric Logarithmic Space

Abstract: The paper's main contributions are a compendium of problems that are complete for symmetric logarithmic space (SL), a collection of material relating to SL, a list of open problems, and an extension to the number of problems known to be SL-complete. Complete problems are one method of studying SL, a class for which programming is non-intuitive. Our exposition helps make the class SL less mysterious and more accessible to other researchers.

A Note on Uniform Circuit Lower Bounds for the Counting Hierarchy

Abstract: A very recent paper by Caussinus, McKenzie, Therien, and Vollmer [CMTV95] shows that ACC0 is properly contained in ModPH, and TC0 is properly contained in the counting hierarchy. Thus, [CMTV95] shows that there are problems in ModPH that require superpolynomialsize uniform ACC0 circuits, and problems in the counting hierarchy that require superpolynomial-size uniform TC0 circuits. The proof in [CMTV95] uses “leaf languages” as a tool in obtaining their separations, and their proof does not immediately yield larger lower bounds for the complexity of these problems. In this paper, we give a simple direct proof of these same separations, and use it to provide “sub-subexponential” size lower bounds on the size of uniform circuits for these problems.

Improved Non-approximability Results for Minimum Vertex Cover with Density Constraints

Abstract: We provide new non-approximability results for the restrictions of the Min Vertex Cover problem to bounded-degree, sparse and dense graphs. We show that, for a sufficiently large B, the recent 16/15 lower bound proved by Bellare et al. [3] extends with negligible loss to graphs with bounded degree B. Then, we consider sparse graphs with no dense components (i.e. everywhere sparse graphs), and we show a similar result but with a better trade-off between non-approximability and sparsity. Finally we observe that the Min Vertex Cover problem remains APX-complete when restricted to dense graph and thus recent techniques developed by Arora et al. [1] for several Max SNP problems restricted to “dense” instances cannot be applied.

On the Computational Complexity of some Classical Equivalence Relations on Boolean Functions

Abstract: The paper analyzes in terms of polynomial time many-one reductions the computational complexity of several natural equivalence relations on Boolean functions which derive from replacing variables by expressions, one of them is the Boolean isomorphism relation. Most of these computational problems turn out to be between co-NP and $\Sigma$ p 2 .

Neither Reading Few Bits Twice nor Reading Illegally Helps Much

Abstract: We first consider the so-called (1, +s)-branching programs in which along every consistent path at most s variables are tested more than once. We prove that any such program computing a characteristic function of a linear code C has size at least 2 Ω(min/s d 1 , d 2 s ), where d1 and d2 are the minimal distances of C and its dual C⊥. We apply this criterion to explicit linear codes and obtain a super-polynomial lower bound for s = o( n logn ) . Then we introduce a natural generalization of read-k-times and (1, +s)-branching programs that we call semantic branching programs. These programs correspond to corrupting Turing machines which, unlike eraser machines, are allowed to read input bits even illegally, i.e. in excess of their quota on multiple readings, but in that case they receive in response an unpredictably corrupted value. We generalize the above-mentioned bound to the semantic case, and also prove exponential lower bounds for semantic read-once nondeterministic branching programs.

An Isomorphism Theorem for Circuit Complexity

Abstract: We show that all sets complete for NC/sup 1/ under AC/sup 0/ reductions are isomorphic under AC/sup 0/-computable isomorphisms. Although our proof does not generalize directly to other complexity classes, we do show that, for all complexity classes C closed under NC/sup 1/-computable many-one reductions, the sets complete for C under NC/sup 0/ reductions are all isomorphic under AC/sup 0/-computable isomorphisms. Our result showing that the complete degree for NC/sup 1/ collapses to an isomorphism type follows from a theorem showing that in NC/sup 1/, the complete degrees for AC/sup 0/ and NC/sup 0/ reducibility coincide. This theorem does not hold for strongly uniform reduction: we show that there are Dlogtime-uniform AC/sup 0/-complete sets for NC/sup 1/ that are not Dlogtime-uniform NC/sup 0/-complete.

The Computational Power of Spiking Neurons Depends on the Shape of the Postsynaptic Potentials

Abstract: Recently one has started to investigate the computational power of spiking neurons (also called \\integrate and re neurons\"). These are neuron models that are substantially more realistic from the biological point of view than the ones which are traditionally employed in arti cial neural nets. It has turned out that the computational power of networks of spiking neurons is quite large. In particular they have the ability to communicate and manipulate analog variables in spatio-temporal coding, i.e. encoded in the time points when speci c neurons \\ re\" (and thus send a \\spike\" to other neurons). These preceding results have motivated the question which details of the ring mechanism of spiking neurons are essential for their computational power, and which details are \\accidental\" aspects of their realization in biological \\wetware\". Obviously this question becomes important if one wants to capture some of the advantages of computing and learning with spatio-temporal coding in a new generation of arti cial neural nets, such as for example pulse stream VLSI. The ring mechanism of spiking neurons is de ned in terms of their postsynaptic potentials or \\response functions\", which describe the change in their electric membrane potential as a result of the ring of another neuron. We consider in this article the case where the response functions of spiking neurons are assumed to be of the mathematically most elementary type: they are assumed to be step-functions (i.e. piecewise constant functions). This happens to be the functional form which has so far been adapted most frequently in pulse stream VLSI as the form of potential changes (\\pulses\") that mimic the role of postsynaptic potentials in biological neural systems. We prove the rather surprising result that in models without noise the computational power of networks of spiking neurons with arbitrary piecewise constant response functions is strictly weaker than that of networks where the response functions of neurons also contain short segments where they increase respectively decrease in a linear fashion (which is in fact biologically more realistic). More precisely we show for example that an addition of analog numbers is impossible for a network of spiking neurons with piecewise constant response functions (with any bounded number of computation steps, i.e. spikes), whereas addition of analog numbers is easy if the response functions have linearly increasing segments.

On the approximability of some NP-hard minimization problems for linear systems

Abstract: We investigate the computational complexity of two classes of combinatorial optimization problems related to linear systems and study the relationship between their approximability properties. In the rst class (Min ULR) one wishes, given a possibly infeasible system of linear relations, to nd a solution that violates as few relations as possible while satisfying all the others. In the second class (Min RVLS) the linear system is supposed to be feasible and one looks for a solution with as few nonzero variables as possible. For both Min ULR and Min RVLS the four basic types of relational operators =, , > and 6 = are considered. While Min RVLS with equations was known to be NP-hard in 27], we established in 2, 6] that Min ULR with equalities and inequalities are NP-hard even when restricted to homogeneous systems with bipolar coeecients. The latter problems have been shown hard to approximate in 8]. In this paper we determine strong bounds on the approximability of various variants of Min RVLS and Min ULR, including constrained ones where the variables are restricted to take bounded discrete values or where some relations are mandatory while others are optional. The various NP-hard versions turn out to have diierent approximability properties depending on the type of relations and the additional constraints, but none of them can be approximated within any constant factor, unless P=NP. Two interesting special cases of Min RVLS and Min ULR that arise in discriminant analysis and machine learning are also discussed. In particular, we disprove a conjecture presented in 57] regarding the existence of a polynomial time algorithm to design linear classiiers (or perceptrons) that use a close-to-minimum number of features.

Modulo Information from Nonadaptive Queries to NP

Abstract: The classes of languages accepted by nondeterministic polynomial-time Turing machines (NP machines, in short) that have restricted access to an NP oracle|the machines can ask k queries to the NP oracle and the answer they receive is the number of queries in the oracle language modulo a number m 2|are considered. It was shown in HT95] that these classes coincide with an appropriate level of the Boolean hierarchy when m is even or k 2m. Here, it is shown that the same holds when m is odd and k 2m, and the level of the Boolean hierarchy is given by k + 3 ? b(k + 2)=mc + (k + b(k + 2)=mc)(mod 2): These results are also generalized to the case when the NP machines are replaced by Turing machines accepting languages of the l th level of the Boolean hierarchy.