Concrete mathematics, a foundation for computer science, by Ronald L. Graham, Donald E. Knuth and Oren Patashnik. Pp 625. £24·95. 1989. ISBN 0-201-14236-8 (Addison-Wesley)

国際会議開催報告:2013 IEEE International Symposium on Information Theory

An investigation is made of relays whose reliability can be described in simple terms by means of probabilities. It is shown that by using a sufficiently large number of these relays in the proper manner, circuits can be built which are arbitrarily reliable, regardless of how unreliable the original relays are. Various properties of these circuits are elucidated.

Reliable Circuits Using Less Reliable Relays

Concentration inequalities have been the subject of exciting developments during the last two decades, and have been intensively studied and used as a powerful tool in various areas. These include convex geometry, functional analysis, statistical physics, mathematical statistics, pure and applied probability theory (e.g., concentration of measure phenomena in random graphs, random matrices, and percolation), information theory, theoretical computer science, learning theory, and dynamical systems.This monograph focuses on some of the key modern mathematical tools that are used for the derivation of concentration inequalities, on their links to information theory, and on their various applications to communications and coding. In addition to being a survey, this monograph also includes various new recent results derived by the authors.The first part of the monograph introduces classical concentration inequalities for martingales, aswell as some recent refinements and extensions. The power and versatility of the martingale approach is exemplified in the context of codes defined on graphs and iterative decoding algorithms, as well as codes for wireless communication.The second part of the monograph introduces the entropy method, an information-theoretic technique for deriving concentration inequalities for functions of many independent random variables. The basic ingredients of the entropy method are discussed first in conjunction with the closely related topic of logarithmic Sobolev inequalities, which are typical of the so-called functional approach to studying the concentration of measure phenomenon. The discussion on logarithmic Sobolev inequalities is complemented by a related viewpoint based on probability in metric spaces. This viewpoint centers around the so-called transportation-cost inequalities, whose roots are in information theory. Some representative results on concentration for dependent random variables are briefly summarized, with emphasis on their connections to the entropy method. Finally, we discuss several applications of the entropy method and related information-theoretic tools to problems in communications and coding. These include strong converses, empirical distributions of good channel codes with non-vanishing error probability, and an information-theoretic converse for concentration of measure.

/pdf/concentration-of-measure-inequalities-in-information-theory-51wfnf6vwi.pdf

Concentration of Measure Inequalities in Information Theory, Communications and Coding

Introduction to finite fields and their applications: Factorization of Polynomials

This paper investigates the extended weight enumerators for the number-theoretic insertion/deletion correcting codes. As a special case, this paper provides the Hamming weight enumerators and cardinalities of the non-binary VT codes.

Weight Enumerators for Number-Theoretic Codes and Cardinalities of Non-binary VT Codes

We propose an efficient encoding algorithm for the binary and non-binary low-density parity-check codes. This algorithm transforms the parity part of the parity-check matrix into a block triangular matrix with low weight diagonal submatrices by row and column permutations in the preprocessing stage. This algorithm determines the parity part of a codeword by block back-substitution to reduce the encoding complexity in the encoding stage. Numerical experiments show that this algorithm has a lower encoding complexity than existing encoding algorithms. Moreover, we show that this algorithm encodes the non-binary cycle codes in linear time.

Efficient Encoding Algorithm of Binary and Non-Binary LDPC Codes Using Block Triangulation.

Luby et al. derived evolution of degree distributions in residual graphs for irregular LDPC code ensembles. Evolution of degree distributions in residual graphs is an important characteristic which is used for finite-length analysis of the expected block and bit error probabilities over the binary erasure channel. In this paper, we derive detailed evolution of degree distributions in residual graphs for irregular LDPC code ensembles with joint degree distributions.

Detailed evolution of degree distributions in residual graphs with joint degree distributions

The number-theoretic codes are a class of codes defined by single or multiple congruences. These codes are mainly used for correcting insertion and deletion errors, and for correcting asymmetric errors. This paper presents a formula for a generalization of the complete weight enumerator for the number-theoretic codes. This formula allows us to derive the weight enumerators and cardinalities for the number-theoretic codes. As a special case, this paper provides the Hamming weight enumerators and cardinalities of the non-binary Tenengolts' codes, correcting single insertion or deletion. Moreover, we show that the formula deduces the MacWilliams identity for the linear codes over the ring of integers modulo $r$.

Weight Enumerators and Cardinalities for Number-Theoretic Codes.

We study LDPC codes for the channel with $2^m$-ary input $\underline{x}\in \mathbb{F}_2^m$ and output $\underline{y}=\underline{x}+\underline{z}\in \mathbb{F}_2^m$. The receiver knows a subspace $V\subset \mathbb{F}_2^m$ from which $\underline{z}=\underline{y}-\underline{x}$ is uniformly chosen. Or equivalently, the receiver receives an affine subspace $\underline{y}-V$ where $\underline{x}$ lies. We consider a joint iterative decoder involving the channel detector and the LDPC decoder. The decoding system considered in this paper can be viewed as a simplified model of the joint iterative decoder over non-binary modulated signal inputs e.g., $2^m$-QAM. We evaluate the performance of binary spatially-coupled MacKay-Neal codes by density evolution. The iterative decoding threshold is seriously degraded by increasing $m$. EXIT-like function curve calculations reveal that this degradation is caused by wiggles and can be mitigated by increasing the randomized window size. The resultant iterative decoding threshold values are very close to the Shannon limit.

Takayuki Nozaki

Papers

Weight Enumerators for Number-Theoretic Codes and Cardinalities of Non-binary VT Codes

Efficient Encoding Algorithm of Binary and Non-Binary LDPC Codes Using Block Triangulation.

Detailed evolution of degree distributions in residual graphs with joint degree distributions

Weight Enumerators and Cardinalities for Number-Theoretic Codes.

Spatially-Coupled Binary MacKay-Neal Codes for Channels with Non-Binary Inputs and Affine Subspace Outputs