It has long been known that the compression redundancy of independent and identically distributed (i.i.d.) strings increases to infinity as the alphabet size grows. It is also apparent that any string can be described by separately conveying its symbols, and its pattern-the order in which the symbols appear. Concentrating on the latter, we show that the patterns of i.i.d. strings over all, including infinite and even unknown, alphabets, can be compressed with diminishing redundancy, both in block and sequentially, and that the compression can be performed in linear time. To establish these results, we show that the number of patterns is the Bell number, that the number of patterns with a given number of symbols is the Stirling number of the second kind, and that the redundancy of patterns can be bounded using results of Hardy and Ramanujan on the number of integer partitions. The results also imply an asymptotically optimal solution for the Good-Turing probability-estimation problem.

Universal compression of memoryless sources over unknown alphabets

Sensor networks potentially feature large numbers of nodes. The nodes can monitor and sense their environment over time, communicate with each other over a wireless network, and process information that they exchange with each other. They differ from data networks in that the network as a whole may be designed for a specific application. We study the theoretical foundations of such large-scale sensor networks. We address four fundamental organizational and operational issues related to large sensor networks: connectivity, capacity, clocks, and function computation. To begin with, a sensor network must be connected so that information can indeed be exchanged between nodes. The connectivity graph of an ad hoc network is modeled as a random graph and the critical range for asymptotic connectivity is determined, as well as the critical number of neighbors that a node needs to connect to. Next, given connectivity, we address the issue of how much data can be transported over the sensor network. We present fundamental bounds on capacity under several models, as well as architectural implications for how wireless communication should be organized. Temporal information is important both for the applications of sensor networks as well as their operation. We present fundamental bounds on the synchronizability of clocks in networks, and also present and analyze algorithms for clock synchronization. Finally, we turn to the issue of gathering relevant information, which sensor networks are designed to do. One needs to study optimal strategies for in-network aggregation of data, in order to reliably compute a composite function of sensor measurements, as well as the complexity of doing so. We address the issue of how such computation can be performed efficiently in a sensor network and the algorithms for doing so, for some classes of functions.

http://cesg.tamu.edu/wp-content/uploads/2015/02/Freris-Kowshik-Kumar-Fundamentals-of-Large-Sensor.pdf

Fundamentals of Large Sensor Networks: Connectivity, Capacity, Clocks, and Computation

We test whether two sequences are generated by the same distribution or by two dierent ones. Unlike previous work, we make no assumptions on the distributions’ support size. Additionally, we compare our performance to that of the best possible test. We describe an eciently-computa ble algorithm based on pattern maximum likelihood that is near optimal whenever the best possible error probability is exp( 14n 2=3 ) using length-n sequences.

/pdf/competitive-closeness-testing-2f5atbs2tr.pdf

Competitive Closeness Testing

How much adversarial noise can protocols for interactive communication tolerate? This question was examined by Braverman and Rao (IEEE Trans. Inf. Theory, 2014) for the case of “robust” protocols, where each party sends messages only in fixed and predetermined rounds. We consider a new class of protocols for interactive communication, which we call adaptive protocols. Such protocols adapt structurally to the noise induced by the channel in the sense that both the order of speaking, and the length of the protocol may vary depending on observed noise. We define models that capture adaptive protocols and study upper and lower bounds on the permissible noise rate in these models. When the length of the protocol may adaptively change according to the noise, we demonstrate a protocol that tolerates noise rates up to 1/3. When the order of speaking may adaptively change as well, we demonstrate a protocol that tolerates noise rates up to 2/3. Hence, adaptivity circumvents an impossibility result of 1/4 on the fraction of tolerable noise (Braverman and Rao, 2014).

https://www.cse.iitd.ac.in/~shweta/papers/AGS.pdf

Adaptive protocols for interactive communication

We study the problems of classification and closeness testing. A classifier associates a test sequence with the one of two training sequences that was generated by the same distribution. A closeness test determines whether two sequences were generated by the same or by different distributions. For both problems all natural algorithms are symmetric—they make the same decision under all symbol relabelings. With no assumptions on the distributions’ support size or relative distance, we construct a classifier and closeness test that require at most ~ O(n 3=2 ) samples to attain then-sample accuracy of the best symmetric classifier or closeness test designed with knowledge of the underlying distributions. Both algorithms run in time linear in the number of samples. Conversely we also show that for any classifier or closeness test, there are distributions that require ~ (n 7=6 ) samples to achieve the n-sample accuracy of the best symmetric algorithm that knows the underlying distributions.

/pdf/competitive-classification-and-closeness-testing-2igbwhr9zb.pdf

Competitive Classification and Closeness Testing

Communication complexity - the minimum amount of communication required - for computing a function of data held by several parties is studied. A communication model where silence is used to convey information is introduced. For this model the worst case and average-case complexities of symmetric functions are studied. For binary-input functions the average- and worst case complexities are determined and the protocols achieving them are described. For functions of nonbinary inputs one-round communication, where each party is restricted to communicate in consecutive stages, is considered and the extra amount of communication required by one- over multiple-round communication is analyzed. For the special case of ternary-input functions close lower and upper bounds on the worst case one-round complexity are provided and protocols achieving them are described. Protocols achieving the average-case one-round complexity for ternary-input functions are also described. These protocols can be generalized to inputs of arbitrary size.

/pdf/silence-based-communication-5fnye1rst8.pdf

Silence-Based Communication

We consider a power-efficient communication model for wireless sensor networks where silence is used to convey information. We study the average-case and worst-case complexities of symmetric functions under this model and describe protocols that achieve them. For binary-input functions, we determine the average complexity. For ternary-input functions, we consider a special type of protocols and provide close lower and upper bounds for their worst-case complexity. We also describe the protocol that achieves the average complexity

Silence Based Communication for Sensor Networks

Recent work has considered encoding a string by separately conveying its symbols and its pattern-the order in which the symbols appear. It was shown that the patterns of independent and identically distributed (i.i.d.) strings can be losslessly compressed with diminishing per-symbol redundancy. In this correspondence, the pattern redundancy of distributions with memory is considered. Close lower and upper bounds are established on the pattern redundancy of strings generated by Hidden Markov models (HMMs) with a small number of states, showing in particular that their per-symbol pattern redundancy diminishes with increasing string length. The upper bounds are obtained by analyzing the growth rate of the number of multidimensional integer partitions, and the lower bounds, using Hayman's theorem

Universal Compression of Markov and Related Sources Over Arbitrary Alphabets

In this paper the pattern redundancy of strings generated by hidden Markov models is bounded over unknown, possibly infinite alphabets, showing in particular that it diminishes to zero when the number of states is sufficiently small.

On the redundancy of HMM patterns

Communication complexity of computing functions using unrestricted communication and one-round communication are compared. In the standard unrestricted communication each party could potentially communicate several times while in one-round communication each party is restricted to communicate at most once. Results on the ratio of these two complexities are provided for symmetric and asymmetric functions under different scenarios. These results are suitably illustrated with examples.

A.K. Dhulipala

Papers

Silence-Based Communication

Silence Based Communication for Sensor Networks

Universal Compression of Markov and Related Sources Over Arbitrary Alphabets

On the redundancy of HMM patterns

Single versus multiple rounds for distributed function computation