Showing papers in "Information & Computation in 1967"
TL;DR: It was found that theclass of context-sensitive languages is learnable from an informant, but that not even the class of regular languages is learningable from a text.
Abstract: Language learnability has been investigated. This refers to the following situation: A class of possible languages is specified, together with a method of presenting information to the learner about an unknown language, which is to be chosen from the class. The question is now asked, “Is the information sufficient to determine which of the possible languages is the unknown language?” Many definitions of learnability are possible, but only the following is considered here: Time is quantized and has a finite starting time. At each time the learner receives a unit of information and is to make a guess as to the identity of the unknown language on the basis of the information received so far. This process continues forever. The class of languages will be considered learnable with respect to the specified method of information presentation if there is an algorithm that the learner can use to make his guesses, the algorithm having the following property: Given any language of the class, there is some finite time after which the guesses will all be the same and they will be correct. In this preliminary investigation, a language is taken to be a set of strings on some finite alphabet. The alphabet is the same for all languages of the class. Several variations of each of the following two basic methods of information presentation are investigated: A text for a language generates the strings of the language in any order such that every string of the language occurs at least once. An informant for a language tells whether a string is in the language, and chooses the strings in some order such that every string occurs at least once. It was found that the class of context-sensitive languages is learnable from an informant, but that not even the class of regular languages is learnable from a text.
3,460 citations
TL;DR: A recognition algorithm is exhibited whereby an arbitrary string over a given vocabulary can be tested for containment in a given context-free language and it is shown that it is completed in a number of steps proportional to the “cube” of the number of symbols in the tested string.
Abstract: A recognition algorithm is exhibited whereby an arbitrary string over a given vocabulary can be tested for containment in a given context-free language. A special merit of this algorithm is that it is completed in a number of steps proportional to the “cube” of the number of symbols in the tested string. As a byproduct of the grammatical analysis, required by the recognition algorithm, one can obtain, by some additional processing not exceeding the “cube” factor of computational complexity, a parsing matrix—a complete summary of the grammatical structure of the sentence. It is also shown how, by means of a minor modification of the recognition algorithm, one can obtain an integer representing the ambiguity of the sentence, i.e., the number of distinct ways in which that sentence can be generated by the grammar. The recognition algorithm is then simulated on a Turing Machine. It is shown that this simulation likewise requires a number of steps proportional to only the “cube” of the test string length.
1,075 citations
TL;DR: The paper is presented in two parts: the first, appearing here, summarizes the major results and treats the case of high transmission rates in detail; the second, to appear in the subsequent issue, treats the cases of low transmission rates.
Abstract: Lower bounds to minimum error probability for block coding on noisy discrete memoryless channels
486 citations
TL;DR: The purpose of this paper is to provide an alternate statement of the Pontryagin maximum principle as applied to systems which are most conveniently and naturally described by matrix, rather than vector, differential or difference equations.
Abstract: Extension of Pontryagin maximum principle and application of matrix minimum principle in solving simple optimal linear regulator problem
408 citations
IBM1
TL;DR: A principal objective of the paper is to establish in the context of abstract algebra, relationships between such concepts as equational and recognizable sets.
Abstract: The concepts of “equational≓ and “recognizable≓ are defined for sets of elements of an arbitrary abstract algebra. Context-free languages and finite-state languages become realizations of the same general concept (of equational sets) when the proper algebra is specified. A principal objective of the paper is to establish in the context of abstract algebra, relationships between such concepts as equational and recognizable sets.
242 citations
TL;DR: New fast methods for decoding certain error-correcting codes by solving certain algebraic equations of small degrees are given.
Abstract: This article gives new fast methods for decoding certain error-correcting codes by solving certain algebraic equations. As described by Peterson (1961) , the locations of a Bose-Chaudhuri Hocquenghem code over a field of characteristic p are associated with the elements of an extension field, GF(pk). The code is designed in such a way that the weighted power-sum symmetric functions of the error locations can be obtained directly by computing appropriately chosen parity checks on the received word. Good methods for computing the elementary symmetric functions from the weighted power-sum symmetric functions have been presented by Berlekamp (1967) . The elementary symmetric functions, σ1, σ2, …, σt are the coefficients of an algebraic equation whose roots are the error locations x t + σ 1 x t − 1 + σ 2 x t − 1 + ⋯ + σ t = 0 Previous methods for finding the roots of this equation have searched all of the elements in GF(pk) ( Chien, 1964 ) or looked up the answer in a large table ( Polkinghorn, 1966 ). We present here improved procedures for extracting the roots of algebraic equations of small degrees.
234 citations
TL;DR: The verification of this minimal time solution to the Firing Squad Synchronization Problem by computer simulation is discussed, and the schema for the mathematical induction proof that the solution works for any length is given.
Abstract: This paper presents a description of a general outline for a minimal time solution to the Firing Squad Synchronization Problem, and a solution of this form which is composed of machines with only eight states. The paper then discusses the verification of this minimal time solution by computer simulation, and gives the schema for the mathematical induction proof that the solution works for any length (the full proof is given in ( Balzer, 1966 )). The final part of the paper discusses some efforts to determine the minimal number of states needed for a minimal time solution. No four state minimal time solution exists. A reasonable set of conditions are presented for which no five state minimal time solution exists. Also, various eight state minimal time solutions exists.
202 citations
TL;DR: Threshold detection as an approximation to optimum detection for weak signals is also defined quantum-mechanically, and threshold detectors for coherent and incoherent fields occupying many modes of a receiver cavity are worked out.
Abstract: Statistical signal detection is formulated quantum-mechanically in terms of choosing one of two density operators as the better description of the state of an ideal receiver after exposure to a field in which a signal may or may not be present. The optimum decision procedure is expressed as a projection operator on the state-space of the receiver. Examples involving the single-mode detection of coherent and incoherent signals are given. Threshold detection as an approximation to optimum detection for weak signals is also defined quantum-mechanically, and threshold detectors for coherent and incoherent fields occupying many modes of a receiver cavity are worked out. The noise in all cases is taken to be thermal radiation described by the Planck law.
166 citations
TL;DR: A number of results on minimum weights in BCH codes are presented, and exact minimum weights have been established for a number of subclasses of NBCH codes.
Abstract: First, some general properties of codes which are invariant under the permutation groups are given. For these codes an interesting relation is given between the minimum weights of dual codes. Secondly, results on the weight distributions of a class of binary cyclic codes are presented. Finally, some results on cross-correlation functions between two maximum-length sequences are introduced.
158 citations
TL;DR: A kind of speedup theorem is proved which is curiously independent of whether the measure of complexity be the size or the number of steps taken by the machines that compute the functions.
Abstract: In this paper, the methods of recursive function theory are used to study the size (or cost or complexity) of machines. A positive result of this study shows that to a remarkable degree, the relative size of two machines is independent of the particular way in which machine size is measured. Another result suggests that in order for programs to be of economical size, the programming language must be powerful enough to compute arbitrary general recursive functions, rather than some restricted subset such as the primitive recursive functions. Finally, a kind of speedup theorem is proved which is curiously independent of whether the measure of complexity be the size or the number of steps taken by the machines that compute the functions.
145 citations
TL;DR: The purpose of this paper is to propose an extension of the domain of study to other algebraic systems to achieve a better Understanding of what takes place in monoids, and to consider the analogs of nondeterministie automata.
Abstract: The study of automata and of context-free languages usually deals with monoids or semigroups. The purpose of this paper is to propose an extension of the domain of study to other algebraic systems. In the process, a better Understanding of what takes place in monoids can be achieved. The basic language is that of the theory of categories. The basic facts are reviewed in sections 2 and 3. The basic ideas of universal :algebra are then introduced. The key notions are that of a "theory" and of algebras belonging to a theory. These ideas were laid down by Lawvere. A streamlined version of Lawvere's theory is given in sections 4-10. This part of the paper is regarded as expository and no proofs are given. Recognizable sets and (deterministic) automata are discussed briefly in sections 11 and 12. In order to consider the analogs of nondeterministie automata, a restriction must be imposed upon the "theories" considered. The normal habitat for this notion is the so-called "linear theories". However, since the main result (Theorem III) is valid only for "free theories" (which are linear), we accept this restriction starting with section 13 and do not introduce linear theories at all. A full treatment of the subject is scheduled to appear in a book by the first of the authors. Relational algebras (section 13) and relational automata (section 14) supply then the analog of nondetermlnistie automata, while the notion
TL;DR: It is possible to test a deterministic pushdown machine to determine if the language it recognizes is regular, and the results show that it should not be trusted to be regular.
Abstract: It is possible to test a deterministic pushdown machine to determine if the language it recognizes is regular.
TL;DR: New and known results in sampling expansions, minimum energy and non-uniform interpolations, and truncation error bounds are presented from a unified point of view of the RKHS.
Abstract: The finite energy Fourier-, Hankel-, sine-, and cosine-transformed bandlimited signals are specific realizations of the abstract reproducing kernel Hilbert space (RKHS). Basic properties of the abstract RKHS are applied to the detailed study of bandlimited signals. The relevancy of the reproducing kernel in extremum problems is discussed. New and known results in sampling expansions, minimum energy and non-uniform interpolations, and truncation error bounds are presented from a unified point of view of the RKHS. Some generalizations and extensions are stated.
TL;DR: A systematic nonlinear code having length 15, minimum distance 5, and 256 code words is given in Boolean form, which is the maximum possible number of words for length 15 and distance 5.
Abstract: A systematic nonlinear code having length 15, minimum distance 5, and 256 code words is given in Boolean form. This is the maximum possible number of words for length 15 and distance 5. The distance spectrum of all pairs of code words is an exact multiple of the weight spectrum of the code words.
TL;DR: This paper considers the synthesis of two-dimensional, periodic, binary and nonbinary sequences (arrays) which exhibit similar properties for their two- dimensional autocorrelation functions and methods of constructing arrays based upon good one-dimensional sequences.
Abstract: Considerable effort has been devoted in the literature to the synthesis of one-dimensional, periodic, binary and nonbinary sequences having small values for their out-of-phase autocorrelation functions. This paper considers the synthesis of two-dimensional, periodic, binary and nonbinary sequences (arrays) which exhibit similar properties for their two-dimensional autocorrelation functions. These arrays may have future application in the areas of optical signal processing, pattern recognition, etc. Various procedures are presented for the synthesis of such arrays. Two perfect binary arrays and an infinite class of perfect nonbinary arrays are given. A class of binary arrays is presented which are the two-dimensional analog of the quadratic residue sequences and are shown to have out-of-phase autocorrelation of −1, or to alternate between +1 and −3. The perfect maps of Gordon are shown to have all values of out-of-phase autocorrelation equal to −1. Other methods of constructing arrays based upon good one-dimensional sequences are also discussed. Synthesis procedures are given for constructing pairs of arrays such that their cross-correlation is identically zero for all shifts and, in addition, individually have good autocorrelation functions. Examples are given for the various synthesis procedures.
TL;DR: It is shown that a simple relationship holds under more general conditions than those implied by Zipf's law.
Abstract: The way in which the number of words occurring once, twice, three times, and so on in a text is related to the vocabulary of the author has been investigated. It is shown that a simple relationship holds under more general conditions than those implied by Zipf's law.
TL;DR: This work examines the question of determining the "best" linear filter, in an expected squared error sense, for a signal generated by stochastic linear differential equation on a Hilbert space and obtains a fully rigorous theory for the finite-dimensional case which does not rely on Ito's Lemma.
Abstract: We examine the question of determining the "best" linear filter, in an expected squared error sense, for a signal generated by stochastic linear differential equation on a Hilbert space. Our results, which extend the development in Kalman and Bucy (1960), rely heavily on the integration theory for Banach-space-valued functions of Dunford and Schwartz (1958). In order to derive the Kalman-Bucy filter, we also need to define and discuss such concepts as stochastic process, covariance, orthogonal increments, Wiener process, and stochastic integral in a Hilbert space context. We do this making extensive use of the ideas in Doob (1953). The two crucial points in our treatment are (1) our definition of the covariance as a bounded linear transformation, and (2) our use of a Fubini4ype theorem involving the interchange of stochastic and Lebesgue integration. As a byproduct, we also obtain a fully rigorous theory for the finite-dimensional case which does not rely on Ito's Lemma (cf. Kushner, 1964). This is of some independent interest. The remainder of the paper is divided into the following sections: 2. Preliminaries; 3. Wiener Processes; 4. Stochastic Integration; 5. An Existence Theorem; 6. The Wiener-Hopf Equation; 7. The Optimal Filter; 8. Concluding Remarks. We introduce some basic preliminary notions, the mos~ important of which is the eovariance of two Hilbert-space-valued random variables, in Section 2. Then, we discuss Wiener processes and construct an infinite-
TL;DR: In this paper it is shown that algorithms exist to determine if a context-free language is a parenthesis language, or if it is equal to the language defined by a given parenthesis grammar.
Abstract: A parenthesis language is a context-free language possessing a grammar in which each application of a production introduces a unique pair of parentheses, delimiting the scope of that production. Parenthesis languages are nontrivial since only one kind of parenthesis is used. In this paper it is shown that algorithms exist to determine if a context-free language is a parenthesis language, or if it is equal to the language defined by a given parenthesis grammar. A possible merit of these results lies in the fact that parenthesis languages are the most general class of languages for which such problems are now known to be solvable; in fact, other problems which are very similar to the one solved here are known to be recursively unsolvable.
TL;DR: An algorithm is presented for finding the loop complexity (sometimes known as “star height≓) of certain regular events whose syntactic monoid is a group.
Abstract: An algorithm is presented for finding the loop complexity (sometimes known as “star height≓) of certain regular events. The algorithm works for those regular events whose syntactic monoid (i.e., semigroup) is a group. The existence of an algorithm that works for all regular events remains an open question.
TL;DR: A new type of automation, called a two-way pushdown automaton is defined and studied which includes relationships with other families, closure and nonclosure results, and decidability properties.
Abstract: In this paper, a new type of automation, called a two-way pushdown automaton is defined and studied. The model is a generalization of a pushdown automaton in that two-way motion is allowed on the input tape which is assumed to have endmarkers. The model is investigated in both the nondeterministic and deterministic cases. A number of basic results are obtained which include relationships with other families, closure and nonclosure results, and decidability properties. Certain special cases are studied such as the cases when the input alphabet has one letter and the device has no endmarkers.
TL;DR: The results prove that under total energy and peak amplitude constraints, the optimal signal must alternate between its peak-power and its zero-power levels.
Abstract: Design of optimal waveforms for optimal estimation or prediction of state of linear dynamical system in presence of Markov and white noise
TL;DR: It is shown that, each family in the sequence of families just named, is a proper subset of the family named after it, and the closure properties of these families under the regular operations and the operation of forming matching-choice sets are considered.
Abstract: The inclusion relations of the families of regular matching-choice sets, meta-linear languages, standard matching-choice sets, and context-free languages are studied. It is shown that, each family in the sequence of families just named, is a proper subset of the family named after it. The closure properties of these families under the regular operations and the operation of forming matching-choice sets are also considered.
IBM1
TL;DR: A correspondence is established between N2, the class of sets of pairs of tapes defined by nondeterministic 2-tape finite automata, and L, theclass of linear context-free languages, and a subclass of L is found, characterized by expressions quite similar to regular expressions.
Abstract: A correspondence is established between N2, the class of sets of pairs of tapes defined by nondeterministic 2-tape finite automata, and L , the class of linear context-free languages. A second correspondence, which is one-one, between N2 and a subclass of L is found. This subclass of L is then characterized by expressions quite similar to regular expressions. It is indicated how to develop analogous characterizations for other subclasses of L . A large portion of this paper appears in the author's doctoral thesis which was presented to the Division of Engineering and Applied Physics, Harvard University. The thesis research was under the supervision of Professor Patrick C. Fischer.
TL;DR: One can construct any finite-state machine as a cascade interconnection of machines whose inputs either permute the states or reset them all to one state using the Krohn-Rhodes theory.
Abstract: One can construct any finite-state machine as a cascade interconnection of machines whose inputs either permute the states or reset them all to one state. Each permutation group needed in the construction is a homomorphic image of a group generated by the action of a set of input sequences on a state subset of the original machine. Proofs of these facts will be given and their application to the Krohn-Rhodes theory described.
TL;DR: I does not hold in general for languages generated by context-sensitive grammars following from the following example: Consider the eontext-~ensitive grammar G2 with nonterminal vocabulary S, #, A, terminal vocabulary a, initial string #S#, and rules.
Abstract: The well-known theorem of Bar-Hillel et al (Bar-Hillel, 1964) to the effect that for every infinite context-free grammar G there exist integers p and q such that for each word a in L(G), the language generated by G, if a contains more than p terminal symbols, then (i) a can be written a -fll~/l~,2f~2 where ~/1~2 is a non-null string and ~fi'2 contains fewer than q terminal symbols, and (ii) flfl,~2"fl2 belongs to L(G) for all n >_ 1 has the following corollary: COROLLARY ( I ) I f L is an infinite language generated by a context-free grammar, then L contains a sequence, {a~}, of strings such that the sequence of lengths {C(a~)} is a ( nontrivial) arithmetic progression The converse of this corollary is not true, as can easily be seen by considering the language L~ = {a~b~a ~ [ n > 1}, which has been shown to be non-context-free in Bar-Hillel (1964) That (I) does not hold in general for languages generated by context-sensitive grammars follows from the following example: EXAMPLE 1 (Due to J Friant, 1966) Consider the eontext-~ensitive grammar G2 with nonterminal vocabulary S, #, A, terminal vocabulary a, initial string #S#, and rules
TL;DR: Nondeterministic Turing machines, under the restriction that each square be written on only a fixed number of times, recognize all and only context free languages.
Abstract: Nondeterministic Turing machines, under the restriction that each square be written on only a fixed number of times, recognize all and only context free languages. The deterministic subclass gives rise to a hierarchical extension of the pushdown deterministic languages. Unambiguity in terms of the machines is the same as grammatical unambiguity.
TL;DR: The enumeration of automata in the natural setting of the power group is reviewed, and the aid of the Power Group Enumeration Theorem is extended to enumerate automata with initial and terminal states.
Abstract: Harary ( 1960, 1964), in a survey of 27 unsolved problems in graphical enumeration, asked for the number of different finite automata. Recently, Harrison (1965) solved this problem, but without considering automata with initial and final states. With the aid of the Power Group Enumeration Theorem (Harary and Palmer, 1965, 1966) the entire problem can be handled routinely. The method involves a confrontation of several different operations on permutation groups. To set the stage, we enumerate ordered pairs of functions with respect to the product of two power groups. Finite automata are then concisely defined as certain ordered pah's of functions. We review the enumeration of automata in the natural setting of the power group, and then extend this result to enumerate automata with initial and terminal states.
TL;DR: It is proved that a set of strings of words forms a categorial language of one type if and only if it forms a category-based language of the other type.
Abstract: There are two types of categorial grammar systems, one studied by Bar-Hillel, Gaifman, and Shamir, and one studied by Lambek. They differ in the type of cancellation allowable. The former has only the rule a(ab) → b and (a/b)b → a . The latter has an added rule that whenfever Xy → z or xY → z , then X → z/y or Y → xz , respectively. The set of strings of words whose categories cancel to s (grammatical sentence) is called the language of the grammar being studied. This paper proves that a set of strings of words forms a categorial language of one type if and only if it forms a categorial language of the other type.
TL;DR: Computational aspects involved in mechanizing the algorithm are discussed in terms of the algorithm's dependence on the solutions of the prediction, filtering, and fixed-point smoothing problems.
Abstract: The algorithm is developed for generating the optimal smoothed estimate x ˆ ( t | t + T ) of the state x(t) of a continuous linear system, where t is continuous time, T is a positive real constant, and t + T is the time of the most recent measurement. A linear matrix differential equation whose solution gives the covariance matrix of the smoothing error x ˆ ( t | t + T ) = x ( T ) - x ˆ ( t | t + T ) is then derived. Computational aspects involved in mechanizing the algorithm are discussed in terms of the algorithm's dependence on the solutions of the prediction, filtering, and fixed-point smoothing problems. The relation of the present results to other studies in estimation theory is indicated.
TL;DR: Algebraic power series in noncommuting variables are obtained as solutions of polynomial systems of equations that can be regarded as context-free grammars with weights.
Abstract: Algebraic power series in noncommuting variables are obtained as solutions of polynomial systems of equations. These systems can be regarded as context-free grammars with weights. Some properties of algebraic power series are derived, in particular their representation as transductions of certain canonical Dyck sets.