Showing papers by "Geoffrey E. Hinton published in 1983"

Optimal perceptual inference

Abstract: When a vision system creates an interpretation of some input datn, it assigns truth values or probabilities to intcrnal hypothcses about the world. We present a non-dctcrministic method for assigning truth values that avoids many of the problcms encountered by existing relaxation methods. Instead of rcprcscnting probabilitics with realnumbers, we usc a more dircct encoding in which thc probability \ associated with a hypotlmis is rcprcscntcd by the probability hat it is in one of two states, true or false. Wc give a particular nondeterministic operator, based on statistical mechanics, for updating the truth values of hypothcses. The operator ensures that the probability of discovering a particular combination of hypothcscs is a simplc function of how good that combination is. Wc show that thcrc is a simple relationship bctween this operator and Bayesian inference, and we describe a learning rule which allows a parallel system to converge on a set ofweights that optimizes its perccptt~al inferences.

Parallel visual computation

Abstract: The functional abilities and parallel architecture of the human visual system are a rich source of ideas about visual processing. Any visual task that we can perform quickly and effortlessly is likely to have a computational solution using a parallel algorithm. Recently, several such parallel algorithms have been found that exploit information implicit in an image to compute intrinsic properties of surfaces, such as surface orientation, reflectance and depth. These algorithms require a computational architecture that has similarities to that of visual cortex in primates.

MASSIVELY PARALLEL ARCHITECTURES FOR Al: METL, THISTLE, AND BOLTZMANN MACHINES

Abstract: It is becoming increasingly apparent that some aspects of intelligent behavior rcquirc enormous computational power and that some sort of massively parallel computing architecture is the most plausible way to deliver such power. Parallelism, rather than raw speed of the computing elements. seems to be the way that the brain gets such jobs done. But even if the need for massive parallelism is admitted, there is still the question of what kind of parallel architecture best fits the needs of various AI tasks. In this paper we will attempt to isolate a number of basic computational tasks that an intelligent system must perform. We will describe several families of massively parallel computing architectures, and we will see which of these computational tasks can be handled by each of these families. In particular, we will describe a new architecture, which we call the Boltzmann machine, whose abilities appear to include a number of tasks that are inefficient or impossible on the other architectures. FAMILIES OF PARALLEL ARCHITECTURES By “massively parallel” architectures, we mean machines with a very large number of processing elements (perhaps very simple ones) working on a single task. A massively parallel system may be complete and self-contained or it may be a special-purpose device, performing some particular task as part of a larger system that contains other modules of a different character. In this paper we will focus on the computation performed by a single parallel module, ignoring the issue of how to integrate a collection of modules into a complete system. * Scott Fahlman i* 3 supported by the Defense Advanced Research Projects Agency, Department of Defense, ARPA Order 3597, monitored by the Air Force Avionics Laboratory under contract F3361581-K-1539. The other two authors are supported by grants from the System Development Foundation. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the U.S. Government. One useful way of classifying these massively parallel architectures is by the type of signal that is passed among the clcmcnts. Fahlman (1982) proposes a division of thcsc systems into three classes: markerpassing, valuc-passing, and message-passing systems. Message-passing systems are the most powerful family, and by far the most complex. They pass around mcssagcs of arbitrary complexity, and perform complex operations on these messages. Such generality has its price: the individual computing clcmcnts are complex, the communication costs are high, and there may be severe contention and traffic congestion problems in the network. Message passing dots not seem plausible as a detailed model of processing in the brain. Such models are being actively studied elsewhere (Hillis, 1981; Hewitt, 1980) and we have nothing more to say about them here. Marker-passing systems, of which NETL (Fahlman, 1979) is an example, arc the simplest family and the most limited. In such systems, the communication among processing elements is in the form of single-bit markers. Each “node” element has the capacity to store a few distinct marker bits (typically 16) and to perform simple Boolean operations on the stored bits and on marker bits arriving from other elements. These nodes are connected by hardware “links” that pass markers from node to node, under orders from an external control computer. The links arc, in effect, dedicated private lines, so a lot of marker traffic can proceed in parallel. A node may be connected to any number of links, and it is the pattern of node-link connections that forms the system’s long-term memory. In NETL, the elements are wired up to form the nodes and links of a semantic network that represents some body of knowledge. Certain common but computation-intensive searches and deductions arc accomplished by passing markers from node to node through the links of this network. A key point about marker-passing systems is that there is never any contention due to message traffic. If many copies of the same marker arrive at a node at once, they are simply OR’ed together. Value-passing systems pass around continuous quantities or numbers and perform simple arithmetic operations on these values. From: AAAI-83 Proceedings. Copyright ©1983, AAAI (www.aaai.org). All rights reserved.

Massively parallel architectures for AI: netl, thistle, and boltzmann machines

Abstract: It is becoming increasingly apparent that some aspects of intelligent behavior require enormous computational power and that some sort of massively parallel computing architecture is the most plausible way to deliver such power. Parallelism, rather than raw speed of the computing elements, seems to be the way that the brain gets such jobs done. But even if die need for massive parallelism is admitted, there is still the question of what kind of parallel architecture best fits the needs of various AI tasks. In this paper we will attempt to isolate a number of basic computational tasks that an intelligent system must perform. We will describe several families of massively parallel computing architectures, and we will see which of diese computational tasks can be handled by each of these families. In particular, we will describe a new architecture, which we call the Boltzmann machine, whose abilities appear to include a number of tasks that are inefficient or impossible on the other architectures.