We survey and summarize the literature on the computational aspects of neural network models by presenting a detailed taxonomy of the various models according to their complexity theoretic characteristics. The criteria of classification include the architecture of the network (feedforward versus recurrent), time model (discrete versus continuous), state type (binary versus analog), weight constraints (symmetric versus asymmetric), network size (finite nets versus infinite families), and computation type (deterministic versus probabilistic), among others. The underlying results concerning the computational power and complexity issues of perceptron, radial basis function, winner-take-all, and spiking neural networks are briefly surveyed, with pointers to the relevant literature. In our survey, we focus mainly on the digital computation whose inputs and outputs are binary in nature, although their values are quite often encoded as analog neuron states. We omit the important learning issues.

/pdf/general-purpose-computation-with-neural-networks-a-survey-of-1ctntc7vc3.pdf

General-purpose computation with neural networks: a survey of complexity theoretic results

Computing with truly asynchronous threshold logic networks

We exhibit a class of symmetric neural networks which synchronous iteration possesses an exponen tial transient length. In fact if {I , ... , n} is the set of nodes we prove the transient length satisfies T 2: 2 n / 3 . For sequential up dating we get the bound f 2: 2 n / 6 • Thi s behavior shows t hat the dynami cs of these class of network s is complex while the steady states are simple: only fixed points or orbits of period 2.

Exponential Transient Classes of Symmetric Neural N etworks for Synchronous and Sequential Updating

We survey some aspects ofthe computational complexity theory ofdiscrete-time and discrete-state Hopfield networks. The emphasis is on topics that are not adequately covered by the existing survey literature, most significantly: 1. the known upper and lower bounds for the convergence times of Hopfield nets (here we consider mainly worst-case results); 2. the power of Hopfield nets as general computing devices (as opposed to their applications to associative memory and optimization); 5’. the complexity of the synthesis (“learning”) and analysis problems related to Hopfield nets as associative memories. *Draft chapter for the forthcoming book The Computational and Learning Complexity of Neural Networks: Advanced Topics (ed. Ian Parberry). ! " # $ ¤ % & ' ( ) * + # ' ( ' , . ( / 0 1 / 2 3 ( ' * 4 ( 5 6 ( ' . 0 7 " 8 9 9 9 8 : % / ( 6 ' ' 7 ; " 8 < " : / 6 = ' . 2 / > * ? 6 @ A B ' , / ( C ' D ( 3 E . ' , ( ' * ' / ' / 6 = / ' , ( ' F 8 G / H H A A I J K L M * N ( / ' ( % / 6 ( ' F 6 / / ( ( ( / 2 J @ J O P Q R L * 4 ( ' 6 E / 5 ' , ' ( ( ' ( C S 6 . 0 T U K L M % / ( 6 1 ' ' , 6 6 ' 2 1 / 2 3 . 0 R U R > 8 R V 8 9 9 9 8 R W * N ( X ' = Y 2 ( C ' D = ' = % 6 ' ( ( ' ( E / 5 T 0 E E % ' 6 / K L M U K M L , ' / 6 F % G * N ( 6 3 1 0 % 6 ' C 1 % C 6 / 2 2 ' / ' ( / 2 2 0 3 / 2 ' 6 E ' S ( / 2 2 / ' , P @ Z [ [ A H ( C ' D * X ' = Y 2 ( C ' D @ A [ , K L L U ¤ , ' / 2 2 F % / ( @ [ A @ A [ , K L L \ ¤ , ' / 2 2 F * , 3 ( / E ( / 2 , / ] ^ % + _ % $ ! # $ ¤ % & ' ( ! * ` # 6 / , ' / ( C ' D ' , a 3 b ' ' ( ( S ' ( ( ' ( C S 6 / ( 6 6 ' 2 ' , / . ' 2 3 1 / 2 3 / E ' < " c W d > e f V % * * % ' , 2 ( S 6 g 2 ' S . * 4 / 0 6 / / ( C ' D 6 / @ [ P A I J @ , 6 / . ' 2 3 1 / 2 3 ' , 6 C S 6 / = ' 2 0 ( ' E / 2 2 0 . ' 3 ( ( * & 2 0 = / D ( S % 6 ( ' ' ( / ( ' ( 2 0 . Y ( C 6 ( / @ h Q H ' , ( C ' D ' ( * ? 6 6 6 ' 2 1 / 2 3 / ( 6 ( / 2 ' . / 3 E ' . = ' 2 0 ( ' E / 2 2 0 . ' 3 ( * ? 6 i P j A I J ' , / ( C ' D Y ( / 6 3 E ' , 6 / . ' 2 3 1 / 2 3 ' , ' ( ( ' ( C S 6 * / ( 0 E k % / 6 ( ' F ( / ( C ' D 6 / / / l L % C 6 6 6 ; " ' < " * ? 6 m | n t x r  y s t v { n o       u v y { u x y s p r q r x n t v x o      w r { q s t u v w u y r q  v r p x ~ x x | u v y n x | x x r r q q ¢ y x q q  z { w y q r q n t r t x n { r |  ~ { | r q ¢ n t r t x n | r q x { | t x q u v r | } x r ~ q x p { r t v x u y q q x n w y |  x | u x | } m ~  v  { £  { }  |  v 

Complexity Issues in Discrete Hopﬁeld Networks

We prove that polynomial size discrete Hopfield networks with hidden units compute exactly the class of Boolean functions PSPACE/poly, i.e., the same functions as are computed by polynomial space-bounded nonuniform Turing machines. As a corollary to the construction, we observe also that networks with polynomially bounded interconnection weights compute exactly the class of functions P/poly, i.e., the class computed by polynomial time-bounded nonuniform Turing machines.

/pdf/the-computational-power-of-discrete-hopfield-nets-with-3ysjq88r81.pdf

M. Tchuente

Papers

Sequential simulation of parallel iterations and applications