Showing papers in "Information & Computation in 1971"
TL;DR: The amplitude-constrained capacity of a scalar Gaussian channel is shown to be achieved by a unique discrete random variable taking on a finite number of values.
Abstract: The amplitude-constrained capacity of a scalar Gaussian channel is shown to be achieved by a unique discrete random variable taking on a finite number of values. Necessary and sufficient conditions for the distribution of this random variable are obtained. These conditions permit determination of the random variable and capacity as a function of the constraint value. The capacity of the same Gaussian channel subject, additionally, to a nontrivial variance constraint is also shown to be achieved by a unique discrete random variable taking on a finite number of values. Likewise, capacity is determined as a function of both amplitude- and variance-constraint values.
542 citations
TL;DR: In this paper explicit formulas for the weight enumerators for several classes of subcodes of the 2nd-order binary Reed-Muller codes are derived and a large set of codes are shown to have the same weight enumerator.
Abstract: In this paper explicit formulas for the weight enumerators for several classes of subcodes of the 2nd-order binary Reed-Muller codes are derived. A large set of the codes are shown to have the same weight enumerators. The classes of codes studied in this paper contain the (0, 2)th-order Euclidean Geometry codes and the codes studied by Berlekamp as subclasses.
375 citations
TL;DR: A new definition of generalized information measures is introduced so as to investigate the finite-parameter estimation problem and yields a class of generalized entropy functions which are useful for treating the error-probability of decision and the other equivocation measures in the same framework.
Abstract: A new definition of generalized information measures is introduced so as to investigate the finite-parameter estimation problem. This definition yields a class of generalized entropy functions which is useful for treating the error-probability of decision and the other equivocation measures such as Shannon's logarithmic measure in the same framework and, in particular, deriving upper bounds to the error-probability. A few of inequalities between these equivocation measures are presented including an extension of Fano's inequality.
201 citations
TL;DR: It is shown that tree automata provide an exact characterization for those GSDT's with a linear relationship between input and output length, and the functional relationship between the length of input and length of output for translations defined by G SDT's is investigated.
Abstract: Two schemes for the specification of translations on a context-free grammar are proposed. The first scheme, called a generalized syntax directed translation (GSDT), consists of a context free grammar with a set of semantic rules associated with each production of the grammar. In a GSDT an input word is parsed according to the underlying context free grammar, and at each node of the tree, a finite number of translation strings are computed in terms of the translation strings defined at the descendants of that node. The functional relationship between the length of input and length of output for translations defined by GSDT's is investigated. The second method for the specification of translations is in terms of tree automata—finite automata with output, walking on derivation trees of a context free grammar. It is shown that tree automata provide an exact characterization for those GSDT's with a linear relationship between input and output length.
177 citations
TL;DR: The concepts of inconsistency, validity, prime implicant and prime implicate are extended to fuzzy logic and various properties of these notions in the context of fuzzy logic are established.
Abstract: In this paper, the fuzzy set ZZadeh (1965)] is viewed as a multivalued logic with a continuum of truth values in the interval Z0, 1]. The concepts of inconsistency, validity, prime implicant and prime implicate are extended to fuzzy logic and various properties of these notions in the context of fuzzy logic are established. It is proved that a formula is valid (inconsistent) in fuzzy logic iff it is valid (inconsistent) in two-valued logic. An algorithm that generates fuzzy prime implicants (implicates) is introduced. A proof of the completeness of this algorithm is also given.
132 citations
[...]
TL;DR: The basic set theory type results for Zadeh's fuzzy sets are shown to carry over and some results on convex fuzzy sets, star-shaped fuzzy set, and arcwise connected fuzzy set results are given.
Abstract: Fuzzy sets are defined as mappings from sets into Boolean lattices. The basic set theory type results for Zadeh's fuzzy sets are shown to carry over. Some results on convex fuzzy sets, star-shaped fuzzy sets, and arcwise connected fuzzy sets are given. Fuzzy sets with “holes,” bounded fuzzy sets, and connected fuzzy sets are also discussed.
108 citations
TL;DR: It is demonstrated that any linear time cost can be eliminated and, in fact, a speed-up by arbitrary positive integer factor k can be attained at an increased structural cost.
Abstract: The general theory of cellular automata is investigated with special attention to structural complexity. In particular, simulation of cellular automata by cellular automata is used to make explicit trade-off relationships between neighborhood size and state-set cardinality. A minimum neighborhood template with d + 1 elements is established for the class of d -dimensional cellular automata. The minimum state set for this class is shown to be the binary state set. The temporal costs, if any, of structural complexity trade-offs are also studied. It is demonstrated that any linear time cost can be eliminated and, in fact, a speed-up by arbitrary positive integer factor k can be attained at an increased structural cost.
89 citations
TL;DR: The problem of constructing minimum-redundancy prefix codes for the general discrete noiseless channel without constraints is solved for unequal code letter costs, provided that the symbols encoded are assumed to be equally probable.
Abstract: The problem of constructing minimum-redundancy prefix codes for the general discrete noiseless channel without constraints is solved for unequal code letter costs, provided that the symbols encoded are assumed to be equally probable. A graphical technique is developed for solving the problem for which the code words are equally probable and are constructed from r symbols where r is greater than or equal to two. A method is given for constructing an optimal exhaustive prefix code. This method is then generalized to the extent that the exhaustive constraint is deleted, thereby resulting in an algorithm, designated ACE for arbitrary symbol cost and equal code word probability, which solves the stated problem.
78 citations
[...]
TL;DR: In 0 L -languages, words are produced from each other by the simultaneous transition of all letters according to a set of production rules; the context is ignored.
Abstract: Summary In 0 L -languages, words are produced from each other by the simultaneous transition of all letters according to a set of production rules; the context is ignored. (i) 0 L -languages are not closed under the operations usually considered. (ii) 0 L -languages over a one-letter alphabet are discussed separately; a characterization is given of a subclass. (iii) 0 L -languages are incomparable with regular sets, incomparable with context-free languages, and strictly included in context-sensitive languages.
70 citations
TL;DR: It has been shown that system identification is equivalent to multihypothesis testing, with a continuum or finite sequence of hypotheses, respectively, for continuous or finite discrete range of θ .
Abstract: Recent results of Middleton and Esposito (1968) and Lainiotis (1969) on single-shot joing detection-estimation for discrete data are extended to the single-shot continuous data case and generalized to joint Bayesian detection-estimation-system identification. Moreover, previous results were generalized to the case of causal estimator. Specifically, it is shown that the above problem constitutes a class of nonlinear mse estimation problems, with the attendant difficulties in realizing the optimal nonlinear estimators. However, by utilizing the adaptive approach, closed form integral expressions are given. These are given in terms of the generalized likelihood ratio gλ(t) , which is a sufficient statistic for Bayes-optimal compound detection. The latter in turn is specified by a continuum (for continuous θ ) θ -conditional likelihood ratious λ(t/θ) each of which is the LR for testing for the model specified by the parameter value θ . The latter LR's are, moreover, given in terms of optimal mse causal estimators. In essence then, it has been shown that system identification is equivalent to multihypothesis testing, with a continuum or finite sequence of hypotheses, respectively, for continuous or finite discrete range of θ .
57 citations
TL;DR: Given the matrix of impulse response sequences of a finite-dimensional discrete-time, linear, constant dynamical system, a state-variable model in the canonical form of Bucy is constructed, an alternate to the Ho-Kalman algorithm.
Abstract: Given the matrix of impulse response sequences of a finite-dimensional discrete-time, linear, constant dynamical system, a state-variable model in the canonical form of Bucy is constructed. The construction is an alternate to the Ho-Kalman algorithm.
TL;DR: It is shown that by utilizing the so-called filter function of transformations the descriptive power of transformational grammars can be preserved unreduced even when their base components are subjected to drastic restrictions.
Abstract: We investigate the effects of placing various restrictions on the base component of a transformational grammar as defined by Chomsky (1965) . It is shown that by utilizing the so-called filter function of transformations the descriptive power of transformational grammars can be preserved unreduced even when their base components are subjected to drastic restrictions.
TL;DR: A broad class of random sampling schemes, for which the sampling intervals are dependent, is constructed and it is shown that this class is “alias free≓ relative to various families of spectra”.
Abstract: This paper deals with the problem of perfect reconstruction of the spectrum of a weakly stationary stochastic process x ( t ) from a set of random samples { x ( t n )}. A broad class of random sampling schemes, for which the sampling intervals are dependent, is constructed. It is shown that this class is “alias free≓ relative to various families of spectra. It is further shown that the alias free property of this class of sampling schemes is invariant under random deletion of samples.
TL;DR: The concepts of structural and behavioral isomorphism on tessellation automata are investigated and certain equivalence relations preserving one or both forms of isomorphic lead to standardizations of neighborhood structure.
Abstract: The concepts of structural and behavioral isomorphism on tessellation automata are investigated. Certain equivalence relations preserving one or both forms of isomorphism lead to standardizations of neighborhood structure. The concepts of blocking and the blocked structure play a central role. A weaker form of behavioral isomorphism is also introduced leading to further simplifications of standard neighborhood structure. Finally, a concept of simulation is investigated.
TL;DR: A method for semantic description of algorithms is proposed, based on the notion of a “tail function” of an algorithm, describing all possible computations defined by this algorithm.
Abstract: A method for semantic description of algorithms is proposed. The basis of the method is the notion of a “tail function” of an algorithm, describing all possible computations defined by this algorithm. The notion of a characteristic equation for an algorithm is introduced and some relations between these equations and the corresponding tail functions are shown. An example, illustrating an application of the method, is given.
TL;DR: All the discrete, memoryless, symmetric channels matched to the Lee metric are derived, and general properties of Lee metric block codes are investigated.
Abstract: Almost all nonbinary codes were designed for the Hamming metric. The Lee metric was defined by Lee in 1958 . Golomb and Welch (1968) and Berlekamp, 1968a , Berlekamp, 1968b have designed codes for the Lee metric. In this paper, we derive all the discrete, memoryless, symmetric channels matched to the Lee metric, and investigate general properties of Lee metric block codes. Finally, a class of cyclic Lee metric codes is defined and the number of information symbols is discussed.
TL;DR: A new proof is presented of the fact that G acts on the eight points as the alternating group, A 8 , and on the 15 points on the general linear group, G ℳ(4, 2) is presented.
Abstract: Let ℳ denote the Mathieu group on 24 points. Let G be the subgroup of ℳ which has three sets of transitivity, the eight points on a Golay code-word (or Steiner octuple), one additional point, and the remaining 15 points. Using elementary results from the subject of algebraic coding theory, we present a new proof of the fact that G acts on the eight points as the alternating group, A 8 , and on the 15 points as the general linear group, G ℳ(4, 2). This result and other properties of the Mathieu groups obtained from it are then used to obtain the symmetry groups of the Nordstrom—Robinson nonlinear (15, 8) code and the linear, cyclic (15, 7) and (21, 12) BCH codes and the (21, 10) dual of a projective geometry code, all of which have distance 5.
TL;DR: In this paper, the properties of reachable sets for linear dynamical systems for specified control sets are discussed and methods of obtaining an admissible control function which transfers an initial state to as near a prescribed target as possible are described.
Abstract: The properties of reachable sets for linear dynamical systems for specified control sets are discussed. Iterative procedures for determining numerical approximations of the reachable set are suggested and methods of obtaining an admissible control function which transfers an initial state to as near a prescribed target as possible is described. The problem of reachability with multiple control constraints is discussed and certain aspects of reachability for time-invariant systems with adjustable parameters is considered.
TL;DR: A linear context-free language which is not acceptable by a finite probabilistic automaton is given, and it is shown that the family of stochastic languages is not closed under concatenation and homomorphism.
Abstract: A linear context-free language which is not acceptable by a finite probabilistic automaton is given, and it is shown that the family of stochastic languages is not closed under concatenation and homomorphism.
TL;DR: A hierarchy of context-free grammars and languages with respect to the index ofcontext-free Grammars is established and the undecidability of the basic problems is proven.
Abstract: A hierarchy of context-free grammars and languages with respect to the index of context-free grammars is established and the undecidability of the basic problems is proven.
TL;DR: In this paper unsolvability of equivalence is shown for a special class of nondeterministic 2-way sequential transducers, and it is shown that deterministic transducers are not so powerful as nondetergetic transducers.
Abstract: A 2-way sequential transducer is a 2-way finite acceptor to which an output structure has been added. Such a device is considerably more complex than the usual 1-way sequential transducer, and it is shown that these transducers map both regular sets and context free languages into the context sensitive languages. Such a transducer is the simplest known which can map a list into its reversal or make duplicate copies of a list. In this paper unsolvability of equivalence is shown for a special class of nondeterministic 2-way sequential transducers, and it is shown that deterministic transducers are not so powerful as nondeterministic transducers. The properties of two-way transductions of regular-sets and context free languages are derived in detail and are shown to be similar to the corresponding properties of context free languages. Also, in this paper the intimate relationship between 2-way sequential transducers and 1-way stack automata is shown by presenting a new grammar which generates exactly the 1-way stack automata languages, and many of the properties of these grammars are derived.
TL;DR: A new method of decoding is presented that utilizes algebraic constraints across streams of convolutionally encoded information sequences to improve the performance of ordinary sequential decoding and over the older hybrid scheme developed by Falconer.
Abstract: Hybrid decoding technique for symmetrical binary input channels, using bootstrap algorithm across convolutionally encoded information streams
TL;DR: In this paper, linear and nonlinear single-error-correcting perfect codes are constructed in this framework by generalizing algebraic methods of Zaremba, with components x i being elements of groups G i which may vary with i.
Abstract: Vectors ( x 1 , x 2 , …, x n ) are considered with components x i being elements of groups G i which may vary with i . Linear and nonlinear single-error-correcting perfect codes are constructed in this framework by generalizing algebraic methods of Zaremba.
TL;DR: It is shown that concatenation, union, and cap are sufficient to characterize context free languages and there are effective procedures for finding a cap expression from a context-free grammar and vice versa.
Abstract: A new operation on languages, called the “cap operation” is introduced, and corresponding “cap expressions” are defined. It is shown that concatenation, union, and cap are sufficient to characterize context free languages. Furthermore there are effective procedures for finding a cap expression from a context-free grammar and vice versa.
TL;DR: It is shown that in some cases one can obtain unambiguity only by increasing complexity, and in any complexity class there are unambiguous languages, all simplest grammars of which are ambiguous.
Abstract: Four of the criteria of complexity of the description of context-free languages by context-free grammars are considered. The unsolvability of the basic problems is proved for each of these criteria. For instance, it is unsolvable to determine the complexity of the language generated by a given grammar, or to find out the simplest grammar, or to decide whether a given grammar is the simplest one and so on. Next, it is shown that in some cases one can obtain unambiguity only by increasing complexity. Namely, for each of the four criteria, in any complexity class there are unambiguous languages, all simplest grammars of which are ambiguous. As one would expect, it is unsolvable whether for an arbitrary grammar G there are unambiguous grammars within the simplest grammars for the language generated by G.
TL;DR: The general uncertainty relation for real time functions in communication theory is derived, where the product of pulse duration and spectral width referred to the positive frequency spectrum only, is not less than 1.1802….
Abstract: The general uncertainty relation for real time functions in communication theory is derived. The product of pulse duration and spectral width referred to the positive frequency spectrum only, is not less than 1.1802…, as compared with 2 in the Heisenberg and Gabor cases. This minimum is reached with a pulse whose time and spectral functions are numerically evaluated.
TL;DR: The purpose of the paper is to prove that the family of languages accepted by finite probabilistic automata is not closed under any of the operations catenation,Catenation closure and homomorphism.
Abstract: The purpose of the paper is to prove that the family of languages accepted by finite probabilistic automata is not closed under any of the operations catenation, catenation closure and homomorphism.
TL;DR: A new class is defined of geometric codes that are majority logic decodable up to their minimum distance using the quasicyclic structure of the codes to describe a very simple decoding procedure.
Abstract: A new class is defined of geometric codes that are majority logic decodable up to their minimum distance. Using the quasicyclic structure of the codes, one describes a very simple decoding procedure.
TL;DR: In this paper, the authors introduced generalized shifted linear codes (SLC) which can achieve a positive rate on arbitrary discrete memoryless channels if Shannon's capacity is positive and the length of the alphabet is less or equal to 5.
Abstract: Summary It was proved by Ahlswede (1971) that codes whose codewords form a group or even a linear space do not achieve Shannon's capacity for discrete memoryless channels even if the decoding procedure is arbitrary Sharper results were obtained in Part I of this paper For practical purposes, one is interested not only in codes which allow a short encoding procedure but also an efficient decoding procedure Linear codes—the codewords form a linear space and the decoding is done by coset leader decoding — have a fairly efficient decoding procedure But in order to achieve high rates the following slight generalization turns out to be very useful: We allow the encoder to use a coset of a linear space as a set of codewords We call these codes shifted linear codes or coset codes They were implicitly used by Dobrushin (1963) This new code concept has all the advantages of the previous one with respect to encoding and decoding efficiency and enables us to achieve positive rate on discrete memoryless channels whenever Shannon's channel capacity is positive and the length of the alphabet is less or equal to 5 (Theorem 311) (The result holds very likely also for all alphabets with a length a = ps, p prime, s positive integer) A disadvantage of the concepts of linear codes and of shifted linear codes is that they can be defined only for alphabets whose length is a prime power In order to overcome this difficulty, we introduce generalized shifted linear codes With these codes we can achieve a positive rate on arbitrary discrete memoryless channels if Shannon's capacity is positive (Theorem 321)
TL;DR: In deterministic zero-sum two-person games, the upper and lower values move towards each other as the information available to the players is increased, but these results fail to hold in an essential way when the game is stochastic.
Abstract: In deterministic zero-sum two-person games, the upper and lower values move towards each other as the information available to the players is increased. This basic fact still holds, with appropriate interpretation, even when the closed loop equations fail to have a unique solution for all pairs of strategies, a hypothesis that would at times be overly restrictive. In particular, if a saddle-point exists for some information pattern, then it remains a saddle-point when the information available is increased. In that case, the additional information can only enable a player to come closer to the ideal of permanent optimality. These results, though, fail to hold in an essential way when the game is stochastic.