On the circuit complexity of sigmoid feedforward neural networks

On the circuit and VLSI complexity of threshold gate comparison

Carver Mead introduced “neuromorphic engineering” [1] as an interdisciplinary approach to the design of biologically inspired neural information processing systems, whereby neurophysiological models of perception and information processing in biological systems are mapped onto analog VLSI systems that not only emulate their functions but also resemble their structure [18]. The motivation for emulating neural function and structure in analog VLSI is the realization that challenging tasks of perception, classification, association and control successfully performed by living organisms can only be accomplished in artificial systems by using an implementation medium that matches their structure and organization.

Neuromorphic learning VLSI systems: a survey

—The paper aims to show that VLSI efficient im-plementations of Boolean functions (BFs) using threshold gates(TGs) are possible. First we detail depth-size tradeoffs for COM-PARISON when implemented by TGs of variable fan-in (∆); aclass of polynomially bounded TG circuits having O ( lgn ⁄ lg∆ )depth and O ( n ⁄ ∆ ) size for any 3 ≤ ∆ ≤ clgn, improves on the pre-vious known size O (n). We then proceed to show how fan-in in-fluences the range of weights and of thresholds, and extend theseresults to F n,m , the class of functions of n variables having mgroups of ones. We conclude that the fan-in could be used by VLSIdesigners for tuning the area-time performances of neural chips.Index terms—neural networks, threshold circuits, Booleanfunctions, F n,m , fan-in, area-time complexity, VLSI. I. I NTRODUCTION I N this paper we shall consider neural networks (NNs) madeof linear threshold gates (TGs), each one computing aBoolean function (BF) f : {0,1} n ➞ {0,1}. If the input vectoris Z = (z

Area-Time Performances of Some Neural Computations

Artificial Neural Networks (ANNs) are often presented as powerful tools for data processing. Nevertheless, ANNs need a theory and consequently an epistemological foundation. Usually scholars in this field seem to believe that the brain theory is an implicit theory for ANNs. But, we still do not have a complete brain theory, and the brain model for ANNs has been only an “inspiration” model. Further, a theory has to explain a set of models or a set of phenomena internally. So, ANN lacks a theory. In this part of the book the author creates and presents a theory of ANNs. Three levels of different complexity are created, in order to simulate a generative path starting from the elementary units up to the more complex ones. The basic concepts of the first level are the node and the connections. At the second level the main concept is the network. At the last level the concept of an artificial organism is the key concept. Explicit rules govern the conversion from one level to another. Further, at each level, sem...

Theory: Foundations of Artificial Neural Networks

Hardware implementation of neuromorphic algorithms is hampered by high degrees of connectivity. Functionally equivalent feedforward networks may be formed by using limited fan-in nodes and additional layers, but this complicates procedures for determining weight magnitudes. No direct mapping of weights exists between fully and limited-interconnect nets. Low-level nonlinearities prevent the formation of internal representations of widely separated spatial features and the use of gradient descent methods to minimize output error is hampered by error magnitude dissipation. The judicious use of linear summations or collection units is proposed as a solution.

/pdf/training-a-limited-interconnect-synthetic-neural-ic-48bkn9vs3w.pdf

Larry A. Akers

Papers

Training a Limited-Interconnect, Synthetic Neural IC