scispace - formally typeset
Search or ask a question
Author

Mark M. Christiansen

Bio: Mark M. Christiansen is an academic researcher from Maynooth University. The author has contributed to research in topics: Rényi entropy & List decoding. The author has an hindex of 14, co-authored 20 publications receiving 412 citations.

Papers
More filters
Journal ArticleDOI
TL;DR: In this article, the authors explore properties and applications of the principal inertia components (PICs) between two discrete random variables and show that they can be used to characterize information-theoretic limits of certain estimation problems.
Abstract: We explore properties and applications of the principal inertia components (PICs) between two discrete random variables $X$ and $Y$ . The PICs lie in the intersection of information and estimation theory, and provide a fine-grained decomposition of the dependence between $X$ and $Y$ . Moreover, the PICs describe which functions of $X$ can or cannot be reliably inferred (in terms of MMSE), given an observation of $Y$ . We demonstrate that the PICs play an important role in information theory, and they can be used to characterize information-theoretic limits of certain estimation problems. In privacy settings, we prove that the PICs are related to the fundamental limits of perfect privacy.

78 citations

Journal ArticleDOI
TL;DR: In this paper, it was shown that the guesswork satisfies a large deviation principle (LDP), which is a lower bound on the expected number of guesses, but one which is not tight in general.
Abstract: How hard is it to guess a password? Massey showed that a simple function of the Shannon entropy of the distribution from which the password is selected is a lower bound on the expected number of guesses, but one which is not tight in general. In a series of subsequent papers under ever less restrictive stochastic assumptions, an asymptotic relationship as password length grows between scaled moments of the guesswork and specific Renyi entropy was identified. Here, we show that, when appropriately scaled, as the password length grows, the logarithm of the guesswork satisfies a large deviation principle (LDP), providing direct estimates of the guesswork distribution when passwords are long. The rate function governing the LDP possesses a specific, restrictive form that encapsulates underlying structure in the nature of guesswork. Returning to Massey's original observation, a corollary to the LDP shows that expectation of the logarithm of the guesswork is the specific Shannon entropy of the password selection process.

72 citations

Journal ArticleDOI
TL;DR: It is shown that, when appropriately scaled, as the password length grows, the logarithm of the guesswork satisfies a large deviation principle (LDP), providing direct estimates of the Guesswork distribution when passwords are long.
Abstract: How hard is it guess a password? Massey showed that that the Shannon entropy of the distribution from which the password is selected is a lower bound on the expected number of guesses, but one which is not tight in general. In a series of subsequent papers under ever less restrictive stochastic assumptions, an asymptotic relationship as password length grows between scaled moments of the guesswork and specific Renyi entropy was identified. Here we show that, when appropriately scaled, as the password length grows the logarithm of the guesswork satisfies a Large Deviation Principle (LDP), providing direct estimates of the guesswork distribution when passwords are long. The rate function governing the LDP possess a specific, restrictive form that encapsulates underlying structure in the nature of guesswork. Returning to Massey's original observation, a corollary to the LDP shows that expectation of the logarithm of the guesswork is the specific Shannon entropy of the password selection process.

55 citations

Journal ArticleDOI
TL;DR: In this article, it was shown that unless $U=V$, there is no general strategy that minimizes the distribution of the number of guesses, but in the asymptote as the strings become long.
Abstract: The guesswork problem was originally motivated by a desire to quantify computational security for single user systems. Leveraging recent results from its analysis, we extend the remit and utility of the framework to the quantification of the computational security of multi-user systems. In particular, assume that $V$ users independently select strings stochastically from a finite, but potentially large, list. An inquisitor who does not know which strings have been selected wishes to identify $U$ of them. The inquisitor knows the selection probabilities of each user and is equipped with a method that enables the testing of each (user, string) pair, one at a time, for whether that string had been selected by that user. Here, we establish that, unless $U=V$ , there is no general strategy that minimizes the distribution of the number of guesses, but in the asymptote as the strings become long we prove the following: by construction, there is an asymptotically optimal class of strategies; the number of guesses required in an asymptotically optimal strategy satisfies a large deviation principle with a rate function, which is not necessarily convex, that can be determined from the rate functions of optimally guessing individual users’ strings; if all users’ selection statistics are identical, the exponential growth rate of the average guesswork as the string-length increases is determined by the specific Renyi entropy of the string-source with parameter $(V-U+1)/(V-U+2)$ , generalizing the known $V=U=1$ case; and that the Shannon entropy of the source is a lower bound on the average guesswork growth rate for all $U$ and $V$ , thus providing a bound on computational security for multi-user systems. Examples are presented to illustrate these results and their ramifications for systems design.

35 citations

Proceedings ArticleDOI
01 Oct 2013
TL;DR: In this paper, a lower bound for the average estimation error based on the marginal distribution of the hidden variable X and the principal inertias of the joint distribution matrix of X and Y is presented.
Abstract: Lower bounds for the average probability of error of estimating a hidden variable X given an observation of a correlated random variable Y, and Fano's inequality in particular, play a central role in information theory. In this paper, we present a lower bound for the average estimation error based on the marginal distribution of X and the principal inertias of the joint distribution matrix of X and Y. Furthermore, we discuss an information measure based on the sum of the largest principal inertias, called k-correlation, which generalizes maximal correlation. We show that k-correlation satisfies the Data Processing Inequality and is convex in the conditional distribution of Y given X. Finally, we investigate how to answer a fundamental question in inference and privacy: given an observation Y, can we estimate a function f(X) of the hidden random variable X with an average error below a certain threshold? We provide a general method for answering this question using an approach based on rate-distortion theory.

30 citations


Cited by
More filters
Journal ArticleDOI

2,415 citations

Posted Content
TL;DR: This work proposes q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by fair resource allocation in wireless networks that encourages a more fair accuracy distribution across devices in federated networks.
Abstract: Federated learning involves training statistical models in massive, heterogeneous networks. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work, we propose q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by fair resource allocation in wireless networks that encourages a more fair (specifically, a more uniform) accuracy distribution across devices in federated networks. To solve q-FFL, we devise a communication-efficient method, q-FedAvg, that is suited to federated networks. We validate both the effectiveness of q-FFL and the efficiency of q-FedAvg on a suite of federated datasets with both convex and non-convex models, and show that q-FFL (along with q-FedAvg) outperforms existing baselines in terms of the resulting fairness, flexibility, and efficiency.

298 citations

BookDOI
01 Jan 2014
TL;DR: This paper shows the first explicit algorithm which can construct strongly k-secure network coding schemes, and it runs in polynomial time for fixed k.
Abstract: We say that a network coding scheme is strongly 1-secure if a source node s can multicast n field elements {m1, · · · ,mn} to a set of sink nodes {t1, · · · , tq} in such a way that any single edge leaks no information on any S ⊂ {m1, · · · ,mn} with |S| = n − 1, where n = mintimax-flow(s, ti) is the maximum transmission capacity. We also say that a strongly h-secure network coding scheme is strongly (h + 1)secure if any h + 1 edges leak no information on any S ⊂ {m1, · · · ,mn} with |S| = n − (h + 1). In this paper, we show the first explicit algorithm which can construct strongly k-secure network coding schemes. In particular, it runs in polynomial time for fixed k.

263 citations