Showing papers in "Information & Computation in 1970"
TL;DR: The concept of information functions of type β (β > 0) is introduced and by means of these information functions the entropies of typeβ are defined, which have a number of interesting algebraic and analytic properties similar to Shannon's entropy.
Abstract: The concept of information functions of type β ( β > 0) is introduced and discussed. By means of these information functions the entropies of type β are defined. These entropies have a number of interesting algebraic and analytic properties similar to Shannon's entropy. The capacity of type β ( β > 1) of a discrete constant channel is defined by means of the entropy of type β . Examples are given for the computation of the capacity of type β , from which the Shannon's capacity can be derived as the limiting case β = 1.
512 citations
TL;DR: The polynomial formulation of generalized ReedMuller codes is somewhat formalized and an extensive study is made of the interrelations between the m- variable approach of Kasami, Lin, and Peterson and the one-variable approach of Mattson and Solomon.
Abstract: The polynomial formulation of generalized ReedMuller codes, first introduced by Kasami, Lin, and Peterson is somewhat formalized and an extensive study is made of the interrelations between the m-variable approach of Kasami, Lin, and Peterson and the one-variable approach of Mattson and Solomon. The automorphism group is studied in great detail, both in the m-variable and in the one-variable language. The number of minimum weight vectors is obtained in the general case. Two ways of restricting generalized ReedMuller codes to subcodes are studied: the nonprimitive and the subfield subcodes. Connections with geometric codes are pointed out and a new series of majority decodable codes is introduced.
283 citations
TL;DR: The class of context-free grammars that can be deterministically parsed in a top down manner with a fixed amount of look-ahead is investigated and it is shown that the equivalence problem is decidable for LL(k) Grammars.
Abstract: The class of context-free grammars that can be deterministically parsed in a top down manner with a fixed amount of look-ahead is investigated. These grammars, called LL(k) grammars where k is the amount of look-ahead are defined and a procedure is given for determining if a context-free grammar is LL(k) for a given value of k . A procedure is given for eliminating the e-rules from an LL(k) grammar at the cost of increasing k by 1. There exist cases in which this increase is inevitable. A procedure is given for obtaining a deterministic push-down machine to recognize a given LL(k) grammar and it is shown that the equivalence problem is decidable for LL(k) grammars. Additional properties are also given.
196 citations
IBM1
TL;DR: A class of discrete noiseless channels having upper and lower bounds on the separation between adjacent nonzero input symbols is considered, and an optimal algebraic block coding scheme for such channels is developed.
Abstract: A class of discrete noiseless channels having upper and lower bounds on the separation between adjacent nonzero input symbols is considered. Recursion relations are derived for determining the number of input sequences which satisfy the constraints for all block lengths, and the asymptotic information rate is calculated. Applications to compaction and synchronization are discussed. An optimal algebraic block coding scheme for such channels is developed.
169 citations
TL;DR: A family of supervised, nonparametric decision rules, based on tolerance regions, is described which includes the k -Nearest Neighbor decision rules when there are two classes.
Abstract: A family of supervised, nonparametric decision rules, based on tolerance regions, is described which includes the k -Nearest Neighbor decision rules when there are two classes. There are two practical reasons for doing so: first, a family of decision rules similar to the k -Nearest Neighbor rules can be specified which applies to a broader collection of pattern recognition problems. This is because in the general class of rules constraints are weakened between the number of training samples required in each training sample set and the respective a priori class probabilities; and, a discrete loss function weighting the importance of the finite number of ways to make a decision error can be introduced. Second, within the family of decision rules based on tolerance regions, there are decision rules which have a property allowing for preprocessing of the training set data resulting in significant data reduction. Theoretical performance for a special case is presented.
168 citations
IBM1
TL;DR: The information theoretical concept of the entropy (channel capacity) of context-free languages and its relation to the structure generating function is investigated and theorems on the convergence parameter of infinite matrices are proved and applied to the evaluation of theropy of certain context- free languages.
Abstract: The information theoretical concept of the entropy (channel capacity) of context-free languages and its relation to the structure generating function is investigated in the first part of this paper. The achieved results are applied to the family of pseudolinear grammars. In the second part, relations between context-free grammars, infinite labelled digraphs and infinite nonnegative matrices are exhibited. Theorems on the convergence parameter of infinite matrices are proved and applied to the evaluation of the entropy of certain context-free languages. Finally, a stochastic process is associated with any context-free language generated by a deterministic labelled digraph, such that the stochastic process is equivalent to the language in the sense that both have the same entropy.
153 citations
TL;DR: In this paper a grey-weighted skeleton is defined for grey-valued continuous and quantized images, and a procedure for the inversion of agrey-weighting skeleton is proposed for obtaining a binary reconstructed image from aGrey-Weighted skeleton.
Abstract: Skeletons have been largely used as descriptors of shape in the field of image processing. Only binary pictures, however, have been considered so far. In this paper a grey-weighted skeleton is defined for grey-valued continuous and quantized images. In order to extend the invertibility property of the skeleton to the grey case, a transformation is defined, which is a generalization of both direct and inverse binary skeleton transformations. By taking advantage of the properties of this transformation, a procedure for the inversion of a grey-weighted skeleton (i.e., for obtaining a binary reconstructed image from a grey-weighted skeleton) is finally proposed.
113 citations
TL;DR: A characterization of L B as the smallest family of languages which contains the bounded context-free languages and which is closed under the operations of union and intersection is proved.
Abstract: Simple matrix languages and right-linear simple matrix languages are defined as subfamilies of matrix languages by putting restrictions on the form and length (degree) of the rewriting rules associated with matrix grammars. For each n ⩾ 1, let L ( n ) [ ℛ ( n ) ] be the class of simple matrix languages [right-linear simple matrix languages] of degree n, and let L = ⋃ n ⩾ 1 L ( n ) [ ℛ = ⋃ n ⩾ 1 ℛ ( n ) ] . It is shown that L ( 1 ) [ ℛ ( 1 ) ] coincides with the class of context-free languages [regular sets] and that L is a proper subset of the family of languages accepted by deterministic linear bounded automata. It is proved that L ( n ) [ ℛ ( n ) ] forms a hierarchy of classes of languages in L [ ℛ ] . The closure properties and decision problems associated with L ( n ) , L , ℛ ( n ) , and ℛ are thoroughly investigated. Let L B [ ℛ B ] be the bounded languages in L [ ℛ ] . It is shown that L B = ℛ B and that most of the positive closure and decision results which are true for bounded context-free languages are carried over in L B . A characterization of L B as the smallest family of languages which contains the bounded context-free languages and which is closed under the operations of union and intersection is proved.
102 citations
TL;DR: A superAFL is a family of languages closed under union with unitary sets, intersection with regular sets, and nested iterated substitution and containing at least one nonunitary set containing all context-free languages.
Abstract: A superAFL is a family of languages closed under union with unitary sets, intersection with regular sets, and nested iterated substitution and containing at least one nonunitary set Every superAFL is a full AFL containing all context-free languages If L is a full principal AFL, then Ŝ∞(L), the least superAFL containing L, is full principal If L is not substitution closed, the substitution closure of L is properly contained in Ŝ∞(L) The indexed languages form a superAFL which is not the least superAFL containing the one-way stack languages If L has a decidable emptiness problem, so does Ŝ∞(L) IfDs is an AFA, L = L(Ds) and Dw is the family of machines whose data structure is a pushdown store of tapes of Ds, then L(Ds) = Ŝ∞(L) if, and only if, Ds is nontrivial If Ds is uniformly erasable and L(Ds) has a decidable emptiness problem, then it is decidable if a member of Dw is finitely nested
96 citations
TL;DR: It is proved that, in general, this extension of web grammar does not increase the generative power of the grammar, but it is useful, for otherwise it is not possible to incorporate negative contextual conditions into the rules, since the context of a given vertex can be unbounded.
Abstract: This paper is concerned with the class of “web grammars,≓ introduced by Pfaltz and Rosenfeld, whose languages are sets of labelled graphs. A slightly modified definition of web grammar is given, in which the rewriting rules can have an applicability condition, and it is proved that, in general, this extension does not increase the generative power of the grammar. This extension is useful, however, for otherwise it is not possible to incorporate negative contextual conditions into the rules, since the context of a given vertex can be unbounded. A number of web grammars are presented which define interesting classes of graphs, including unseparable graphs, unseparable planar graphs and planar graphs. All the grammars in this paper use “normal embeddings≓ in which the connections between the web that is written and the host web are conserved, so that any rewriting rule affects the web only locally.
87 citations
TL;DR: This paper treats two questions about the finite Karhunen-Loeve expansion: how many samples should the wave vector contain and how can the eigenvectors of the autocorrelation matrix be estimated?
Abstract: The finite Karhunen-Loeve expansion has found application recently in pattern recognition and communication theory. Basically, the finite expansion can be considered as a two-step process. First, we take a finite number of samples from a waveform and form a wave vector. Secondly, we expand this vector in the eigenvectors of the autocorrelation matrix. In this paper, we treat two questions. 1. How many samples should the wave vector contain? 2. How can we accurately estimate the eigenvectors of the autocorrelation matrix? We also present a fast algorithm for calculating the eigenvectors of an autocorrelation matrix.
TL;DR: In this article, the authors derived formulas for the number of codewords of each weight in several classes of subcodes of the second-order Reed-Muller codes, including subclasses whose weight enumerators were previously found by Kasami (1967, 1969) and Berlekamp(1968a, b).
Abstract: In this paper we obtain formulas for the number of codewords of each weight in several classes of subcodes of the second order Reed-Muller codes. Our formulas are derived from the following results: (i) the weight enumerator of the second order RM code, as given by Berlekamp-Sloane (1970) , (ii) the MacWilliams-Pless identities, (iii) a new result we present here (Theorem 1), (iv) the Carlitz-Uchiyama (1957) bound, and, (iv′) the BCH bound. The class of codes whose weight enumerators are determined includes subclasses whose weight enumerators were previously found by Kasami (1967–1969) and Berlekamp(1968a, b) .
TL;DR: For context-free grammars, all of the following restrictive devices are equivalent with respect to generative power: a regular control language, programming, a set of matrices, and periodic time-variance.
Abstract: For context-free grammars, all of the following restrictive devices are equivalent with respect to generative power: a regular control language, programming, a set of matrices, and periodic time-variance. This leads to several new characterizations of the family of recursively enumerable languages. Eg., every recursively enumerable language is generated by a context-free grammar, where the set of productions available at the ith step of a derivation is a periodic function of i.
TL;DR: The problem of minimizing the number of sets of states required for Knuth's parsing algorithm is solved by showing it is equivalent to that of finding the minimal representation for an incompletely specified finite automaton.
Abstract: In Knuth (1965) the problem of minimizing the number of sets of states required for his parsing algorithm is raised as an open question. This question is discussed by among others, Lewi (1968) , Korenjak (1969) , Early (1970) , and Loeckx (1970) . We solve the problem by showing it is equivalent to that of finding the minimal representation for an incompletely specified finite automaton. A solution may thus be obtained using the known methods for the latter. 1 This result may be viewed in contrast to Pager, 1969 , Pager, 1970a where it is shown that it is not generally possible to make optimizations of this kind.
TL;DR: The purpose of this book is to present a mathematical theory of the class of machines known as Perceptrons, carefully formulated and focuses on the theoretical capabilities and limitations of these machines.
Abstract: The purpose of this book is to present a mathematical theory of the class of machines known as Perceptrons. The theory is carefully formulated and focuses on the theoretical capabilities and limitations of these machines. It is a remarkable book. Not only do the authors formulate a new and fundamental conceptual framework, but they also fill in the details using strikingly ingenious mathematical techniques. They ask some novel questions and find some difficult answers. The most striking of these will be presented in Section 2. The authors address the book to three classes of readers:
TL;DR: Pushdown automata serve as a base for the description of four basic parsing techniques for general phrase-structure Grammars (Chomsky O-type grammars).
Abstract: Pushdown automata serve as a base for the description of four basic parsing techniques for general phrase-structure grammars (Chomsky O-type grammars).
TL;DR: A nonstatistical approach to extrapolation is proposed and examined, which leads to an optimal extrapolation function π*(x; D), which is studied when the sequences (x, y), (xi, yi) are, in fact, independent and identically distributed random vectors.
Abstract: Prompted by the inadequacies of the now traditional characterization of chance and uncertainty through the Kolmogorov axioms for probability and the relative frequency interpretation of probability, we propose and examine a nonstatistical approach to extrapolation. The basic problem is the association of a real number y to a sequence of real numbers x in such a manner that the pair (x, y) conforms with a set of data sequences D = (xi, yi), {iti} = 1, M, our prior knowledge of the data source, and our objectives. Our aim is to so define the activity of extrapolation that we can derive extrapolations with only minimal assumptions about the data source. While we are free to define the human activity of extrapolation to suit ourselves, the data source functions independently of our wishful or metaphysical thinking. The basic principle we adhere to is that the extrapolation of x is a function of only those yi for which x is similar to or close to xi; extrapolate the output of a system by examination of the outputs of similar systems. This vague sentiment is clarified and formalized through ten axioms and leads to an optimal extrapolation function π*(x; D). The performance of π* is then studied, both for very large and very small sample sizes (M), when the sequences (x, y), (xi, yi) are, in fact, independent and identically distributed random vectors.
TL;DR: A syntax is a category of strings and derivations between them and the semantic domain is a categories of sets and functions.
Abstract: A syntax is a category of strings and derivations between them. The semantic domain is a category of sets and functions. An interpretation is a cofunctor from
TL;DR: This paper explicitly proves the more general result and shows that the proof easily follows from the techniques used in the previous work and indicates in a rigorous mathematical way how the hypotheses may be changed using some results for the decomposition of supermartingales.
Abstract: For a general stochastic signal in white noise absolute continuity is proved and the Radon-Nikodym derivative is given. These results were stated in a previous paper (Duncan 1968). Independent of the absolute continuity result, a modification is proved for the hypothesis with signal present. This paper is a sequel to an earlier paper by the author (Duncan 1968) where likelihood functions were obtained for diffusion process signals. While the general result was noted there, in this paper we explicitly prove the more general result and show that the proof easily follows from the techniques used in the previous work. We shall also indicate in a rigorous mathematical way how the hypotheses may be changed using some results for the decomposition of supermartingales.
TL;DR: A new orthogonal series representation and a new Orthogonal integral representation for harmonizable stochastic processes are presented and it is shown that the former is more efficient than the latter.
Abstract: This paper presents a new orthogonal series representation and a new orthogonal integral representation for harmonizable stochastic processes. It also demonst
TL;DR: It is shown that if a set is recognizable by a multi-tape TM with no more than R(n) tape reversals, where n is the input length, then the set isRecognized by a 1-t tape off-line TM within time T(n).
Abstract: The number of tape reversals required for the recognition of a set of inputs by a 1-tape Turing machine (TM) has been proposed before as a measure of complexi It is shown that if a set is recognizable by a multi-tape TM with no more than R(n) tape reversals, where n is the input length, then the set is recog It is also shown that if a set is recognizable by a multi-tape TM within time T(n), then the set is recognizable by a 1-tape off-line TM with no more than Upper bounds on the number of reversals necessary for the recognition of a number of well-known classes of languages are also obtained.
TL;DR: It is proved that there exists no algorithm (in the sense of recursive function theory) which applies to any context sensitive grammar and which allows one to compute the entropy of the language generated by a contextsensitive grammar.
Abstract: It is proved that there exists no algorithm (in the sense of recursive function theory) which applies to any context sensitive grammar and which allows one to This suggests that there exists no general calculation method to compute the entropy of the language generated by a context sensitive grammar.
TL;DR: A context-sensitive grammar G is said to be CS(k) iff a particular kind of table-driven parser, T k ( G ) , exists, and a class of parsers T ¯ k (G ) is defined, which accepts exactly the sentences of G.
Abstract: A context-sensitive grammar G is said to be CS(k) iff a particular kind of table-driven parser, T k ( G ) , exists. Corresponding to each T k ( G ) , we define a class of parsers T ¯ k ( G ) . T k ( G ) is itself a T ¯ k ( G ) . The main results are: 1. Any processor T ¯ k ( G ) for a CS(k) grammar G accepts exactly the sentences of G. 2. The set of languages generable by CS(k) grammars is exactly the set of languages accepted by deterministic linear-bounded automata (DLBA's). 3.(a) It is undecidable whether there exists any k ⩾ 0 such that an arbitrary CSG is CS(k). (b) For every fixed k ⩾ 0, there is no algorithm that will decide if G is CS(k) and also construct T k ( G ) if it exists. 4. For any DLBA M, algorithms are given to (i) construct a CS(k) grammar GM that generates the language accepted by M, and (ii) construct a processor T ¯ 1 ( G M ) . 5. CS(k) grammars are unambiguous. 6. The sentences of a CS(k) grammar can be parsed in a time proportional to the length of their derivations.
TL;DR: One of Shannon's intuitive schemes for estimating the Entropy of printed English is generalized here to discrete parameter, discrete range random fields.
Abstract: Random fields entropy estimation technique taking into account higher than immediately adjacent spatial dependencies
Journal Article•
TL;DR: It is proved that this equation has no solutions for n > 2, q > 3 (q a prime power) in the case of two errors and that the Golay (23, 12) code is the only nontrivial perfect 3-error-correcting code over any alphabet GF(q).
Abstract: There are two well-known necessary conditions for the existence of a perfect error-correcting code. Combination of these leads to a Diophantine equation. It is proved that this equation has no solutions for n > 2, q > 3 (q a prime power) in the case of two errors. Furthermore we prove that the Golay (23, 12) code is the only nontrivial perfect 3-error-correcting code over any alphabet GF(q).
TL;DR: A new class of synchronizable error-correcting codes which are derivable from cyclic codes by restraining some of their information symbols are presented, called subset codes, which appear to be the most efficient available.
Abstract: This paper presents a new class of synchronizable error-correcting codes which are derivable from cyclic codes by restraining some of their information symbols. They are called subset codes because each code constitutes a subset of code words in the parent cyclic code. The ability of the subset codes to detect and correct loss of synchronism, or slip, in both noiseless and noisy channels is examined. In particular, given a t-error-correcting cyclic code, there exists a subset code that can correct any combination of slip and additive error whose total magnitude does not exceed t. Moreover, when the received word is in synchronism, the decoder automatically corrects any additive error pattern the parent cyclic code could correct. For channels in which slip is small and occurs infrequently, the subset codes appear to be the most efficient available. In other situations they are generally efficient and in addition their decoding procedures are straightforward. The manner in which the information symbols are restrained is the same as for the extended cyclic codes of Bose and Caldwell [1], and Weldon [2]. However, unlike the extended cyclic codes which have redundant symbols added to both ends of each code word, the block length of the subset codes is the same as the parent cyclic codes.
TL;DR: A positive coding theorem and a converse is proved for this channel and a deterministic code (Reed-Solomon code) and bounded discrepancy decoding is proved.
Abstract: A mathematical model is proposed for a postal service which occasionally loses letters. A positive coding theorem and a converse is proved for this channel. The coding theorem is proved by utilizing a deterministic code (Reed-Solomon code) and bounded discrepancy decoding.
TL;DR: A direct proof of the closure of Wiener's orthogonal set { G n } in the class L 2 ( Ω, B, μ ) is presented and an application to stochastic differential equations with memory is discussed in some detail.
Abstract: In this paper a direct proof of the closure of Wiener's orthogonal set { G n } in the class L 2 ( Ω, B, μ ) is presented in Section 2. It In the last section an application of these results to stochastic differential equations with memory is discussed in some detail.