Showing papers presented at "Information Theory Workshop in 2005"

Brief history of quantum cryptography: a personal perspective

Abstract: Quantum cryptography is the only approach to privacy ever proposed that allows two parties (who do not share a long secret key ahead of time) to communicate with provably perfect secrecy under the nose of an eavesdropper endowed with unlimited computational power and whose technology is limited by nothing but the fundamental laws of nature. This essay provides a personal historical perspective on the field. For the sake of liveliness, the style is purposely that of a spontaneous after-dinner speech. It will be seen that the IEEE played a ubiquitous role in this tale

Universal similarity

Abstract: We survey a new area of parameter-free similarity distance measures useful in data-mining, pattern recognition, learning and automatic semantics extraction. Given a family of distances on a set of objects, a distance is universal up to a certain precision for that family if it minorizes every distance in the family between every two objects in the set, up to the stated precision (we do not require the universal distance to be an element of the family). We consider similarity distances for two types of objects: literal objects that as such contain all of their meaning, like genomes or books, and names for objects. The latter may have literal embodyments like the first type, but may also be abstract like "red" or "christianity:' For the first type we consider a family of computable distance measures corresponding to parameters expressing similarity according to particular features between pairs of literal objects. For the second type we consider similarity distances generated by Web users corresponding to particular semantic relations between the (names for) the designated objects. For both families we give universal similarity distance measures, incorporating all particular distance measures in the family. In the first case the universal distance is based on compression and in the second case it is based on Google page counts related to search terms. In both cases experiments on a massive scale give evidence of the viability of the approaches.

Stopping set analysis for Hamming codes

Abstract: In the 2004 Shannon Lecture, McEliece presented an expression for the number of stopping sets of size three in a Hamming code. In this paper, we investigate how this number depends on the parity-check matrix used in the decoding process. First, we present basic results on stopping set enumerators for block codes in general. Next, we focus on stopping set enumerators for Hamming codes. Our main result is a parity-check matrix of relatively small size for which the number of stopping sets of size three equals the number of codewords of weight three in the Hamming code.

Fisher information matrix and hyperbolic geometry

Abstract: The Fisher information matrix induces a metric on parametric spaces of families of probability density functions. We analyse here the family of normal distributions showing how hyperbolic geometry arises naturally from the Fisher information metric.

How to attain the ordinary channel capacity securely in wiretap channels

Abstract: It is known that a message can be transmitted safely against any wiretap per via a noisy channel without a secret key if the coding rate is less than the so-called secrecy capacity Cs, which is usually smaller than the channel capacity C. In order to remove the loss C - Cs, we propose a multiplex coding scheme with plural independent messages. In this paper, it is shown that the proposed multiplex coding scheme can attain the channel capacity as the total rate of the plural messages and the perfect secrecy for each message. The coding theorem is proved by extending Hayashi's proof, in which the coding of the channel resolvability is applied to the wiretap channel

On extractors, error-correction and hiding all partial information

Abstract: Randomness extractors (Nisan and Zuckerman, 1996) allow one to obtain nearly perfect randomness from highly imperfect sources randomness, which are only known to contain "scattered" entropy. Not surprisingly, such extractors have found numerous applications in many areas of computer science including cryptography. Aside from extracting randomness, a less known usage of extractors comes from the fact that they hide all deterministic functions of their (high-entropy) input (Dodis and Smith, 2005): in other words, extractors provide certain level of privacy for the imperfect source that they use. In the latter kind of applications, one typically needs extra properties of extractors, such as invertibility, collision-resistance or error-correction. In this abstract we survey some of such usages of extractors, concentrating on several recent results by the author (Dodis et al., 2004 and Dodis and Smith, 2005). The primitives we survey include several flavors of randomness extractors, entropically secure encryption and perfect one-way hash functions. The main technical tools include several variants of the leftover hash lemma, error correcting codes, and the connection between randomness extraction and hiding all partial information. Due to space constraints, many important references and results are not mentioned here; interested reader can find those in the works of Dodis et al. (2004) and Dodis and Smith (2005)

Finite-length scaling of irregular LDPC code ensembles

Abstract: We investigate the finite-length scaling methodology for irregular LDPC code ensembles when transmission takes place over the binary erasure channel (BEC). We first show how the necessary computations, namely the covariance evolution and the computation of the finite-length shift, can be accomplished in the irregular case. We then investigate how the obtained approximation can be used to predict the performance of irregular code ensembles and to optimize the degree distributions for finite-length codes.

Interleaved Reed-Solomon codes in concatenated code designs

Abstract: Interleaved Reed-Solomon codes allow the correction of errors beyond half the minimum code distance if the errors are not distributed independently in the received signal but occur in bursts. Therefore, these codes are mainly considered for applications in channels, that cause correlated error patterns, i.e., error bursts. However, they can also be quite interesting for memoryless channels causing independent random errors, if they are applied in concatenated code designs. We present such concatenated codes with several outer Reed-Solomon codewords and demonstrate the gain, that can be obtained by interleaved Reed-Solomon decoding in comparison to independently decoding the several words of the underlying Reed-Solomon codes.

Computing the Levenshtein distance of a regular language

Abstract: The edit distance (or Levenshtein distance) between two words is the smallest number of substitutions, insertions, and deletions of symbols that can be used to transform one of the words into the other. In this paper we consider the problem of computing the edit distance of a regular language (also known as constraint system), that is, the set of words accepted by a given finite automaton. This quantity is the smallest edit distance between any pair of distinct words of the language. We show that the problem is of polynomial time complexity. We distinguish two cases depending on whether the given automaton is deterministic or nondeterministic. In the latter case the time complexity is higher.

Provably unbreakable hyper-encryption in the limited access model

Abstract: Encryption is a fundamental building block for computer and communications technologies. Existing encryption methods depend for their security on unproven assumptions. We propose a new model, the limited access model for enabling a simple and practical provably unbreakable encryption scheme. A voluntary network of tens of thousands of computers each maintain and update random pages, and act as page server nodes. A sender and receiver share a random key K. They use K to randomly select the same PSNs and download the same random pages. These are employed in groups of say 30 pages to extract one time pads common to S and R. Under reasonable assumptions of an adversary's inability to monitor all PSNs, and easy ways for S and R to evade monitoring while downloading pages, hyper encryption is clearly unbreakable. The system has been completely implemented

Decoding of second order Reed-Muller codes with a large number of errors

Abstract: Second order Reed-Muller codes are considered over a binary symmetric channel. We present a modified version of V.M Sidel'nikov and A.S. Pershakov algorithm, Problemy Peredachi Informatsii 1992, that has complexity of order n/sup 2/log(n). Experimental results show that the algorithm corrects most error patterns of weight up to n/2(1-e) given that e exceeds n-1/3. This outperforms other decoding algorithms known for RM codes. Decoding performance for known algorithms has been evaluated and the results correspond to asymptotic performance for these algorithms.

Bounds on the entropy of patterns of I.I.D. sequences

Abstract: Bounds on the entropy of patterns of sequences generated by independently identically distributed (i.i.d.) sources are derived. A pattern is a sequence of indices that contains all consecutive integer indices in increasing order of first occurrence. If the alphabet of a source that generated a sequence is unknown, the inevitable cost of coding the unknown alphabet symbols can be exploited to create the pattern of the sequence. This pattern can in turn be compressed by itself. The bounds derived here are functions of the i.i.d. source entropy, alphabet size, and letter probabilities. It is shown that for large alphabets, the pattern entropy must decrease from the i.i.d. one. The decrease is in many cases more significant than the universal coding redundancy bounds derived in prior works. The pattern entropy is confined between two bounds that depend on the arrangement of the letter probabilities in the probability space. For very large alphabets whose size may be greater than the coded pattern length, all low probability letters are packed into one symbol. The pattern entropy is upper and lower bounded in terms of the i.i.d. entropy of the new packed alphabet. Correction terms, which are usually negligible, are provided for both upper and lower bounds.

Fast correlation attacks against stream ciphers and related open problems

Abstract: Fast correlation attacks have been considerably improved recently, based on efficient decoding algorithms dedicated to very large linear codes in the case of a highly noisy channel. However, a better adaptation of these techniques to the concrete involved stream ciphers is still an open issue

On the capacity of erasure relay channel: multi-relay case

Abstract: We consider here a single sender-destination multi-relay channel. The links connecting the nodes are supposed to be erasure where symbols are received correctly without any error, or lost. We consider that the nodes are not able to use any interference cancellation mechanism. The interference might be suppressed through using separated physical channel or thought a time-sharing mechanism. This model is realistic for many practical scenarios in the context of wireless networks. In previous works, the capacity region of broadcast erasure channels as well as the capacity of the single-sender relay channel (under degraded and non-degraded hypothesis) has been derived. This paper extends the previous results to the more general case of multi-relay channels. We derive the cut-set bound for a general (stationary ergodic) multi-relay erasure channel, and we show that it can be reached through a practical linear coding scheme based on MDS codes.

Further results on mapping functions

Abstract: In a previous paper as presented by Tarable et al. (2004), the authors introduced the concept of mapping functions as a tool to cope with the problem of collision-free memory mappings in turbo and LDPC parallel decoder implementations. In this paper, we address some implementation issues of the original solution, define a dual problem, which can be of high interest for practical implementations, and give the generalization of mapping functions to N > 2 parallel decoders.

On polyalphabetic block codes

Abstract: A polyalphabetic (or mixed) block code is a set of codewords of finite length, where every symbol of a codeword belongs to its own alphabet. In contrast to previous publications we consider a general case, where we do not assume any algebraic structure of the alphabets and the codes. Upper and lower bounds on the cardinality of a polyalphabetic code with given Hamming distance are obtained. Some constructions of polyalphabetic codes are suggested based on known codes. Encoding and decoding of the polyalphabetic codes, obtained in this way, can be done using encoding and decoding algorithms for the mother code. Using this constructions, codes are obtained, that reach the upper Singleton type bound.

Interference management via capacity-achieving codes for the deterministic broadcast channel

Abstract: We motivate the consideration of deterministic broadcast channel coding as an interference management technique in wireless scenarios. We address practical coding strategies for such channels and discuss two approaches. The first relies upon enumerative source coding and can be applied for any deterministic broadcast channel problem as the first step in pipelined encoding for vertex rates. The second approach addresses a wireless interference management scenario and is a complete, practical, capacity-achieving strategy that dualizes the Luby transform code construction and encoding/decoding algorithms. This results in the first practical, nontrivial, capacity achieving code construction for the deterministic broadcast channel.

Dynamic access policies for unconditionally secure secret sharing schemes

Abstract: Secret sharing schemes are cryptographic primitives that can be employed in any situation where it is desirable to distribute information relating to secret data amongst a number of entities. Following a secure initialization process, a secret sharing scheme normally has a fixed access policy (specified by an access structure). However in dynamic environments it is likely that access policies will be subject to change: for example entities may wish to join the scheme, entitles may be expelled from the scheme or security thresholds may change. Rather than undergo a costly re-initialization process, there have been several different proposals for designing secret sharing schemes that permit dynamic access policies. We reviewed various existing models and proposals, and identify several open problems

The multivariate merit factor of a Boolean function

Abstract: A new metric, the multivariate merit factor (MMF) of a Boolean function, is presented, and various infinite recursive quadratic sequence constructions are given for which both univariate and multivariate merit factors can be computed exactly. In some cases these constructions lead to merit factors with non-vanishing asymptotes. A formula for the average value of 1/MMF is derived and a characterisation of the MMF in terms of cryptographic differentials is discussed.

Universal estimation of information measures

Abstract: In this presentation, the author gives an overview of the state of the art in universal estimation of: entropy; divergence; mutual information with emphasis on recent algorithms we have proposed with H. Cai, S. Kulkarni and Q. Wang. These algorithms converge to the desired quantities without any knowledge of the statistical properties of the observed data, under several conditions such as stationary-ergodicity in the case of discrete processes, and memorylessness in the case of analog data. A sampling of the literature in this topic is given below.

Unconditionally private and reliable communication in an untrusted network

Abstract: Most successful attacks that occur today against computer networks are targeted against the computers connected to the network, but not against the routers. At the BlackHat 2005 conference it was shown that potential attacks against routers are far from hypothetical. Current TCP/IP protocols (even including IPSEC) do not have the resilience to deal with routers taken over by the adversary. We survey the research done in the area of how to communicate reliably and/or privately. In the presence of different types of adversary, for example a malicious (Byzantine) one. Evidently, if the adversary can control all nodes (routers) in the network, no solution exists. The nodes that can be attacked by the adversary can be described using a threshold, or by what is called an adversary structure. The type of networks studied are point-to-point or broadcast/multicast

Boneh-Shaw fingerprinting and soft decision decoding

Abstract: Collusion-secure codes are used for digital fingerprinting and for traitor tracing. In both cases, the goal is to prevent unauthorised copying of copyrighted material, by tracing at least one guilty user when illegal copies appear. The most well-known collusion-secure code is due to Boneh and Shaw (1995/98). In this paper we improve the decoding algorithm by using soft output from the inner decoder, and we show that this permits using significantly shorter codewords.

Ergodic theorems for algorithmically random sequences

Abstract: We extend the recurrence time theorem, which is originally derived by Wyner and Ziv, to individual Martin-Lof random sequences. We also present a simple, alternative proof of the algorithmic Shannon-McMillan-Breiman theorem that was first proved by V'yugin.

Design of non-systematic turbo codes for universal source controlled channel decoding

Abstract: Non-systematic turbo codes have a large potential advantage over systematic codes when coding redundant sequences. However, to utilize this potential universally, when sequence statistics are unknown in advance, they need to be properly designed. We consider design criteria for non-systematic turbo codes that can allow universal utilization of redundancy in the encoded messages. We apply these criteria to decoding of independently identically distributed (i.i.d.) nonuniform sequences, and of piecewise stationary memoryless sequences. Simulation results are presented for both classes for rate 1/2 and 1/3 codes. They demonstrate that with proper designs, full (or almost full) utilization of the redundancy can be achieved for these source classes even universally, and the performance with unknown statistics is (almost) as good as that with initially known statistics.

Steepest descent as message passing

Abstract: It is shown how steepest descent (or steepest ascent) may be viewed as a message passing algorithm with "local" message update rules. For example, the well-known backpropagation algorithm for the training of feedforward neural networks may be viewed as message passing on a factor graph. The factor graph approach with its emphasis on "local" computations makes it easy to combine steepest descent with other message passing algorithms such as the sum/max-product algorithms, expectation maximization, Kalman filtering/smoothing, and particle filters. As an example, parameter estimation in a state space model is considered. For this example, it is shown how steepest descent can be used for the maximization step in expectation maximization.

Rate-compatible path-pruned convolutional codes and their applications on channels with insertion, deletion and substitution errors

Abstract: A concatenated system is presented to achieve unequal error protection (UEP) on a channel with insertion/deletion/substitution (IDS) errors. By using a feedback-mapping encoder scheme, the path-pruning of a convolutional code is achieved to periodically generate a specific block code, which can be exploited to correct insertion or deletion errors. This paper presents a general means to obtain a path-pruned convolutional code with specific trellis structure by using the puncturing and pruning processes. One application combining the rate-compatible path-pruned convolutional code, partial interleaving, and Reed-Solomon code was implemented to evaluate its performance on the IDS channel.

Temporary assumptions - quantum and classical

Abstract: Long-term security is achieved by protocols which use computational assumptions only during the execution of the protocol and become information theoretically secure afterwards. Coin flipping protocols and zero knowledge arguments are examples for protocols achieving long-term security. In this work we consider this regime between computational security and information theoretic security. A security model for long-term security is sketched and the class K of all two argument functions which can be computed with long-term security is characterised. Furthermore it is shown that the class Q of all two argument functions which can be computed using quantum cryptography is strictly contained in K. The characterisation of K is a generalisation of a result of Kushilevitz (1992), where he characterises the two argument functions which can securely be computed in presence of an unbounded passive adversary. The result in the quantum case additionally relies on the impossibility results of (D. Mayers, 1997) , (H.-K. Lo and H. R Chau, 1996) and the impossibility of quantum coin flipping result of Kitaev which is published in (A. Ambainis et al., 2004)

Maximum entropy and the Edgeworth expansion

Abstract: For sums of i.i.d. random variables the maximum entropy distribution with respect to the first moments fixed is compared with the Edgeworth expansion. It is demonstrated that the Edgeworth expansion can and shall be considered as a linear extrapolation of the maximum entropy distribution. The coefficients in the Edgeworth expansion can be used as a first approximation for numerical calculation of the maximum entropy distribution.

Bounds on the performance of sphere decoding of linear block codes

Abstract: A sphere decoder searches for the closest lattice point within a certain search radius. The search radius provides a tradeoff between performance and complexity. We derive tight upper bounds on the performance of sphere decoding of linear block codes. The performance of soft-decision sphere decoding on AWGN channels as well as that of hard-decision sphere decoding on binary symmetric channels is analyzed.

Bit commitment from weak non-locality

Abstract: So-called non-local boxes, which have been introduced as an idealization n different respects - of the behavior of entangled quantum states, have been known to allow for unconditional bit commitment between the two involved parties. We show that, actually, any possible non-local correlation which produces random bits on both sides can be used to implement bit commitment, and that this holds even when the parties are allowed to delay their inputs to the box. Since a particular example is the behavior of an EPR pair, this resource allows for implementing unconditionally secure bit commitment as long as the parties cannot entangle their Qbits with any other system