Showing papers in "Information & Computation in 1966"
TL;DR: It is shown that the random elements as defined by Kolmogorov possess all conceivable statistical properties of randomness and can equivalently be considered as the elements which withstand a certain universal stochasticity test.
Abstract: Kolmogorov has defined the conditional complexity of an object y when the object x is already given to us as the minimal length of a binary program which by means of x computes y on a certain asymptotically optimal machine. On the basis of this definition he has proposed to consider those elements of a given large finite population to be random whose complexity is maximal. Almost all elements of the population have a complexity which is close to the maximal value. In this paper it is shown that the random elements as defined by Kolmogorov possess all conceivable statistical properties of randomness. They can equivalently be considered as the elements which withstand a certain universal stochasticity test. The definition is extended to infinite binary sequences and it is shown that the non random sequences form a maximal constructive null set. Finally, the Kollektivs introduced by von Alises obtain a definition which seems to satisfy all intuitive requirements.
1,228 citations
TL;DR: Two apparently divergent areas of inquiry should give rise to the same problem, namely, that of describing the infinite history of finite automata, and it is this problem to which the remainder of this paper will address itself.
Abstract: Bfichi (1962) has given a decision procedure for a system of logic known as \" the Sequential Calculus,\" by showing that each well formed formula of the system is equivalent to a fornmla that, roughly speaking, says something about the infinite input history of a finite automaton. In so doing he managed to answer an open question that was of concern to pure logicians, some of whom had no interest in the theory of automata. Muller (1963) came upon quite similar concepts in studying a problem in asynchronous switching theory. The problem was to describe the behavior of an asynchronous circuit tha t does not reach any stability condition when starting from a certain state and subject to a certain input condition. Many different things can happen, since there is no control over how fast various parts of the circuit react with respect to each other. Since at no time during the presence of that input condition will the circuit reach a terminal condition, it will be possible to describe the total set of possibilities in an ideal fashioll only if an infinite amount of time is assumed for tha t input condition. Neither Biichi's Sequential Calculus nor ~Iuller's problem of asynchronous circuitry will be described further here. I t is interesting tha t two such apparently divergent areas of inquiry should give rise to the same problem, namely, that of describing the infinite history of finite automata. I t is this problem to which the remainder of this paper will address itself. I t will be recalled that a well known basic theorem in the theory of
553 citations
TL;DR: A number of results about deterministic languages (languages accepted by pushdown automata with no choice of moves) are established, and several problems are shown to be recursively unsolvable.
Abstract: A number of results about deterministic languages (languages accepted by pushdown automata with no choice of moves) are established. In particular, o (1) each deterministic language is unambiguous. (2) the complement of each deterministic language is a deterministic language. (3) numerous operations which preserve deterministic languages (for example, intersection with a regular set) are obtained. (4) several problems are shown to be recursively unsolvable.
228 citations
TL;DR: It is shown that when given an arbitrarily long array of such machines and a command is given at one end at t = 0, it is possible to cause all the machines to go to one terminal state, all at once, at time t = 2n − 2 where n is the number of machines in the array.
Abstract: A 16 state machine is proposed as an optimum solution to the “Firing Squad Synchronization Problem.” It is shown that when given an arbitrarily long (but finite) array of such machines and a command is given at one end at t = 0, it is possible to cause all the machines to go to one terminal state, all at once, at time t = 2n − 2 where n is the number of machines in the array. The code book of the state transitions of the machine is so arranged to cause the array to progressively divide itself into 2k equal parts, where k is an integer and an increasing function of time. The end machines in each partition assume a special state so that when the last partition occurs, all the machines have for both neighbors machines at this state. This is made the only condition for any machine to assume terminal state.
217 citations
TL;DR: Multitape Turing machines which can use their storage tapes only as counters or as pushdown stores are investigated and it is shown that without restrictions on computing time or amount of tape used that only six distinct classes of sets of strings (languages) are produced.
Abstract: Multitape Turing machines which can use their storage tapes only as counters or as pushdown stores are investigated. The memory access restrictions are produced by regarding the machines as small computers (as in the formalism of Wang) and by restricting the instruction repertoires. Relationships are given linking machines which only accept or reject inputs and machines which emit output sequences as a function of their input. It is shown that without restrictions on computing time or amount of tape used that only six distinct classes of sets of strings (languages) are produced by the above memory access restrictions.
117 citations
TL;DR: The basic definitions of Probabilistie finite automata are adopted here (with certain modifications) to keep this paper within reasonable limits and to omit the proofs of some of the theorems.
Abstract: Probabilistie finite automata (p.a.) as introduced in this paper are intended to generalize deterministic and nondeterministie finite automata (f.a.) in the sense of Rabin and Scott (1959). Although the notion of p.a. was mentioned in a popular book by Ashby (1958) no precise definition has been given in literature except for a recent paper by Rabin (1963). Another a t tempt to generalize the f.a. in the sense of Moore (1956) was made by Carlyle (1961) and a particular form of p.a. is investigated by Tsetslin (1961) and Davis (1961), but all the above papers t reat the p.a. from a different point of view than that assumed by Rabin (1963) and here. An at tempt to define the p.a. in the sense of this paper, and to investigate some of their properties, was already made by the author in his (unpublished) master's thesis undertaken at the Hebrew University of Jerusalem; still, as our point of view is very close to that of Rabin (1963) we shall adopt here (with certain modifications) the basic definitions of tha t paper. In order to keep this paper within reasonable limits we shall omit the proofs of some of the theorems.
98 citations
TL;DR: A self-reproducing universal array is presented with simple programming made possible by using as basic unit a finite automaton which can execute an internal program of up to 20 instructions.
Abstract: von Neumann and Thatcher have shown that one may construct self-reproducing universal arrays using as basic cells finite automata with only 29 states. The simplicity of the components necessitates complex programming. We present a self-reproducing universal array with simple programming. This is made possible by using as basic unit a finite automaton which can execute an internal program of up to 20 instructions.
88 citations
TL;DR: Exact quantum-mechanical expressions, which give the well-known Shannon's formulas in the classical limit case, are obtained and the effect of the quantum nature of electromagnetic field is revealed, the paradox of infinite capacity in the absence of noise being eliminated.
Abstract: The negentropy principle of information (Brillouin's principle) is formulated in a positive form in order to express the amount of information received by a physical system in terms of physical entropy. This principle is applied to the problem of information transmission by the electromagnetic field and information characteristics of both wide-band and narrow-band channels (capacity and minimum energy per unit of information) are calculated on the base of statistical thermodynamics. The usual information-theoretical derivation is also given. Exact quantum-mechanical expressions, which give the well-known Shannon's formulas in the classical limit case, are obtained. On the other hand, the effect of the quantum nature of electromagnetic field is revealed, the paradox of infinite capacity in the absence of noise being eliminated. Problems concerning the number of degrees of freedom, polarization states, and nonequilibrium noise are also discussed.
69 citations
TL;DR: The improper random process s ( t) = ∑ n = − ∞ ∞ α n δ ( t − t n ) where the impulse occurrence times t n constitute a stationary point process and where the intensity modulating coefficients α n constitutes a stationary random process, independent of the point process t n.
Abstract: Improper random process where impulse occurrence times constitutes stationary point process, noting autocorrelation function and spectral density function
62 citations
TL;DR: Randomly sampled random processes, discussing sampling time statistics and stationary point processes are discussed in this paper, where the authors also discuss the relationship between stationary point process and random sampled random process and time complexity.
Abstract: Randomly sampled random processes, discussing sampling time statistics and stationary point processes
60 citations
TL;DR: Using this result, conditions under which pushdown transducers and linear bounded transducers preserve contest sensitive languages are given and the basic result is applied to show that certain rewriting systems generate context sensitive languages instead of arbitrary recursively enumerable sets.
Abstract: A basic result which gives a condition under which a (possibly length-decreasing) homomorphism preserves a contest. sensitive language is presented. Using this result, conditions under which pushdown transducers and linear bounded transducers preserve contest sensitive languages are given. The basic result is also applied to show that certain rewriting systems generate context sensitive languages instead of arbitrary recursively enumerable sets. Of special interest is the result that if each rule in a rewriting system has a terminal letter on its right side, then the language generated is context free.
TL;DR: A linear boundedTransducer is a linear bounded automaton (pushdown automaton) with outputs that maps (a) L sub 1 onto L sub 2 or (b) LSub 1 into L sub2 nontrivially.
Abstract: : A linear bounded transducer (pushdown transducer) is a linear bounded automaton (pushdown automaton) with outputs Answers to the following two problems are derived: (1) If S is a linear bounded transducer or a pushdown transducer, and L is context sensitive, context free, or regular; is S(L) some well known type of set (2) Does there exist a decision procedure to determine for arbitrary sets L sub 1 and L sub 2, both context sensitive or both context free, whether there exists a linear bounded transducer (pushdown transducer) which maps (a) L sub 1 onto L sub 2 or (b) L sub 1 into L sub 2 nontrivially (Author)
TL;DR: An extension of Liapunov's direct method is developed for the investigation of discontinuous dynamical systems using a positive definite function which is constant or decreasing along the trajectories of the system when no pulses are emitted and decreasing during pulse emission.
Abstract: A class of discontinuous dynamical systems is introduced as a generalization of Pulse Frequency Modulation Systems. An extension of Liapunov's direct method is developed for the investigation of their stability. This is done through the choice of a positive definite function which is constant or decreasing along the trajectories of the system when no pulses are emitted and decreasing during pulse emission. The method is also used for the investigation of periodic solutions and a number of examples are presented dealing with the solution of specific problems.
TL;DR: Using the properties of parity-check matrices of binary linear codes a tree-search program is given which finds quasi-perfect codes and yielded eighteen new quasi- perfect double error-correcting codes.
Abstract: Using the properties of parity-check matrices of binary linear codes a tree-search program is given which finds quasi-perfect codes. Fixing the number of check digits and the number of errors to be corrected, the program will find one quasi-perfect code for each block length that such a code exists. The efficiency of the search technique is demonstrated by the fact that the program yielded eighteen new quasi-perfect double error-correcting codes.
TL;DR: The problem of designing an optimum controller based on a quadratic error measure is considered for a distributed parameter system with boundary controls and solutions are obtained for the class of distributed parameter systems described by the N -dimensional diffusion equation.
Abstract: The problem of designing an optimum controller based on a quadratic error measure is considered for a distributed parameter system with boundary controls. Solutions to this problem are obtained for the class of distributed parameter systems described by the N -dimensional diffusion equation. Methods for calculating optimum controls and trajectories are also discussed.
TL;DR: Results include synthesis procedures for constructing the sequential machine when the relation is "reasonably given" and several characterizations of sequential relations are given as well as connections with other families of relations studied in automata theory.
Abstract: Suppose that one has some system with inputs drawn from a set Z* which produces outputs in some set Lx*. The system is assumed to have sonm state set S. Suppose that all possible experiments are run on the system ill the sense that we can tabulate a relation R of all inpuVoutput pairs which the system can produce. (We will vary the starting state.) A problem of some importance in systems theory (Zadeh and Desoer, 1963) is to synthesize a system which has as its relation R where R has been \"given\" in advance. In this paper, we study sequential relations which are the relations which arise in this manner when the system is a finite sequential machine. Our main results include synthesis procedures for constructing the sequential machine when the relation is \"reasonably given.\" Also several characterizations of sequential relations are given as well as connections with other families of relations studied in automata theory. This study was stimulated by a lecture given by Professor 5i. A. Aizermann in July 1964 at the University of California, Berkeley. Professor Aizermann described a method for the synthesis of sequential machines by a \"question-answer\" procedure. Since the technical details of the method were omitted, we derived a method which satisfied the general characteristics mentioncd by Professor Aizernmnn. Recently, the original algorithm has become available (see Tal, 1964). I t turns out that the two methods have very little in common. Our method is • 7, presented as an appendix to this ]oa~dr. , Our method synthesizes a sequential function f by constructing the minimal sequential machine which computes f; this machine has a dis-
TL;DR: A consistency technique for finding associations between parts of patterns is described, and results of computer simulation are given.
Abstract: A consistency technique for finding associations between parts of patterns is described, and results of computer simulation are given.
TL;DR: The purpose of this paper is to explore the connection between the length of storage sequences and the amount of circuitry necessary to reMize the corresponding functions and a new method for the generation of boolean functions set forth here.
Abstract: In an earlier paper, Maurer and Rhodes (1965) proved the following result: Let G be a finite simple nonabelian group and f: G" --> G a map of the set of n-tuples of elements of G into G. Then there is an integer m and an m-tuple t ( t I , t 2 , . . . , t m ) of elements of G 0 { 1 , . . , n } such that for all g = (g l , . . . , g , ) C G ~, f ( g ) --E ( t , g) , where E ( t , g) is the product gl p . . . g~' C G of elements g t C G defined by letting for eachj C {1, . . . , m} g j = t~ if tj C G org j = gtj i f t~ C {1, --,n}. While this result is of some intrinsic algebraic interest, the motivation behind the result was its application to automata theory. I t is the purpose of this paper to explore this connection, in particular, we set forth here a new method for the generation of boolean functions. Currently, the maior method used for that purpose is one developed by Shannon which, while well-known, can require in hardware realizations circuitry too cumbersome for some applications. The alternative method set forth here involves the use of a fixed simple nonabelian group machine (which can be realized in hardware by n flip-flops, where 2" exceeds the order of the group involved) and, for each boolean function, a storage sequence, the length of which depends upon the boolean function and the group machine used. A few examples will convince the reader that there is little correlation between the length of storage sequences and the amount of circuitry necessary to reMize the corresponding functions
TL;DR: The fundamental question of when a PSM is equivalent in some sense to a deterministic machine, perhaps with random devices attached to output states, is considered and various tests involving finitely many random variables are devised.
Abstract: : The concept of probabilistic sequential machines (PSM), a generalization of Rabin's concept of probabilistic automata, is defined. Such diverse devices as unreliable digital computers, slot machines, and chemical cells are presented as examples of PSM. Using the examples as motivation, various kinds of equivalences between machines are discussed. The fundamental question of when a PSM is equivalent in some sense to a deterministic machine, perhaps with random devices attached to output states, is considered. Finally, various tests involving finitely many random variables are devised for each of the kinds of equivalences between PSM and for reduction, if possible, to deterministic machines. One of the tests is a further generalization of the Moore bound for deterministic machines than has previously appeared in the literature.
TL;DR: In this paper some simple methods are given for predicting the possibility of singular detection of known signals in additive gaussian noise, which are useful in the formulation of suitable mathematical models.
Abstract: A mathematical model that leads to singular detection (i.e., detection with arbitrarily low probability of error) is usually a poor model of the physical problem. However, in general, it has been difficult to establish the possibility of singular detection without a detailed evaluation of the probability of error. Such an evaluation is difficult because it requires the solution of various integral equations, the computation of eigenvalues, etc. In this paper some simple methods are given for predicting the possibility of singular detection of known signals in additive gaussian noise. Such methods are useful in the formulation of suitable mathematical models.
TL;DR: Straightforward and direct methods are presented for finding the probabilities of a recurrent event defined over a t -state, r th-order Markov chain, applicable to radar detection, beam splitting problems and the detection of nuclear tracks.
Abstract: Straightforward and direct methods are presented for finding the probabilities of a recurrent event defined over a t -state, r th-order Markov chain. A special case of this problem is that of the determination of the probabilities of events in a Bernoulli sequence. The recurrent event may be either simple or compound, nonoverlapping or overlapping. Consideration is given to stationary and nonstationary Markov chains. The probabilities determined are for: (a) the occurrence of exactly k events in n trials (b) the occurrence of the k th event on the n trial (c) the occurrence of one or more events in n trials To solve for the above probabilities a transformation is made of the general problem stated to the simpler problem of finding the corresponding probabilities for a set of states defined over an s state, first-order Markov chain. The probabilities of the set of states of the reduced problem are found by the use of forward differential equations developed in the past for a continuous time stochastic birth process. Backward differential equations can also be used. The results are applied to radar detection. They are also applicable to beam splitting problems and the detection of nuclear tracks.
TL;DR: Previous formulas for the power spectra of random digital FM signals are generalized to include the possibility of periodically choosing random modulating signals and two interesting special cases are evaluated.
Abstract: Previous formulas for the power spectra of random digital FM signals are generalized to include the possibility of periodically choosing random modulating signals. A large class of practical FM schemes are included in this model. To illustrate the utility of the general formulas we evaluate in detail two interesting special cases. In the first case the modulating random signals are white gaussian and in the second they are pulses occurring at random in a fixed interval T. The latter represents pulse position modulation by FM.
TL;DR: A pattern recognition system has been formulated with the framework of sequential decision theory and application of the work to computer recognition of handprinted and handwritten English characters is examined in detail.
Abstract: A pattern recognition system has been formulated with the framework of sequential decision theory. Recognition of multiple pattern class with features properly ordered is considered. The system can switch from supervised learning to unsupervised learning to continuously improve the performance. Application of the work to computer recognition of handprinted and handwritten English characters is examined in detail.
TL;DR: It is shown that the best random codes consist of strategy function sequences whose elements are selected independently, and the required distribution of the strategy functions is for the present class of channels specified by ah instead of the apparent ah parameters.
Abstract: The form of capacity achieving input distribution is specified for a class of finite state channels with side information available to the transmitter. It is shown that the best random codes consist of strategy function sequences whose elements are selected independently. It is a result of Theorem 4 (which is of independent interest) that the required distribution of the strategy functions is for the present class of channels specified by ah instead of the apparent ah parameters, so that a random selection of optimal codes is practically possible. As an application, an extremely simple capacity expression for finite state galois additive noise channels is obtained, and it is shown that the value of capacity will not increase if the transmitter is supplied with information about the channel states.
TL;DR: Many fields of interest under current research involve dynamical systems, and their stability examined as a special case of problems of stability in dynamics.
Abstract: Many fields of interest under current research involve dynamical systems. In these fields dynamical systems can be formulated, and their stability examined as a special case of problems of stability in dynamics.
TL;DR: In this paper it is shown that (1) the capacity of the product channel is the sum of the capacities of the component channels, and (2) the "strong converse" holds for the product channels if it holds for each of the components.
Abstract: In this paper we shall consider the product or parallel combination of channels, and show that (1) the capacity of the product channel is the sum of the capacities of the component channels, and (2) the "strong converse" holds for the product channel if it holds for each of the component channels. The result is valid for any class of channels (with or without memory, continuous or discrete) provided that the capacities exist. "Ciapacity" is defined here as the supremum of those rates for which arbitrarily high reliability is achievable with block coding for sufficiently long delay. Let us remark here that there are two ways in which "channel capaci ty" is commonly defined. The first definition takes the channel capacity to be the suprenmm of the "information" processed by the channel, where "information" is the difference of the input "uncer ta inty" and the "equivocation" at the output. The second definition, which is the one we use here, takes the channel capacity to be the maximum "error free ra te ." For certain classes of channels (e.g., memoryless channels, and finite state indecomposable channels) it has been established that these two definitions are equivalent. In fact, this equivalence is the essence of the Fundamental Theorem of Information Theory. For such channels, (1) above follows directly. The second definition, however, is applicable to a broader class of channels than the first. One very important such class are time-continuous channels.
TL;DR: A model is proposed for recognizing the individual characters in running text without preliminary segmentation using a hierarchy of transformations which retain the two dimensional form of the original pattern.
Abstract: A model is proposed for recognizing the individual characters in running text without preliminary segmentation. The model employs a hierarchy of transformations which retain the two dimensional form of the original pattern. Using a computer simulation preliminary experiments on a limited alphabet indicate the potential power of the model to recognize textual material. Numerical results are given for a range of character spacings and variation in some parameters of the model.
IBM1
TL;DR: It is shown that the sufficient conditions as originally stated by Shannon are indeed also the necessary conditions for the range of symbol lengths which are of interest.
Abstract: Shannon (1948) showed that, by a proper choice of the conditional probabilities of the symbols in a discrete noiseless channel of the finite-state type (which possesses certain inherent constraints), the entropy of symbols on such a channel could be maximized. To date these known sufficient conditions on the conditional probabilities have been assumed to also be the necessary conditions. In this brief paper it is shown that the sufficient conditions as originally stated by Shannon are indeed also the necessary conditions for the range of symbol lengths which are of interest.
TL;DR: The failure of the conjecture is established by showing N(L) context free for the nonregular set L = {aibidk | i ≠ j V j ≠ k}ce:italic>.
Abstract: For a set L of words over an alphabet lacking the letter c, let N(L) = {xcy | x, y in L Λ x ≠ y}. Haines has shown that if L is regular, N(L) is context free. The converse has been widely conjectured. If true, it would allegedly support a neat argument to show that English is not context free. The failure of the conjecture is established by showing N(L) context free for the nonregular set L = {aibidk | i ≠ j V j ≠ k}ce:italic>.