Showing papers in "IEEE Transactions on Information Theory in 2021"

Non-Bayesian Activity Detection, Large-Scale Fading Coefficient Estimation, and Unsourced Random Access With a Massive MIMO Receiver

Abstract: In this paper, we study the problem of user activity detection and large-scale fading coefficient estimation in a random access wireless uplink with a massive MIMO base station with a large number $M$ of antennas and a large number of wireless single-antenna devices (users). We consider a block fading channel model where the $M$ -dimensional channel vector of each user remains constant over a coherence block containing $L$ signal dimensions in time-frequency. In the considered setting, the number of potential users $K_{\text {tot}}$ is much larger than $L$ but at each time slot only $K_{a} \ll K_{\text {tot}}$ of them are active. Previous results, based on compressed sensing, require that $K_{a}\le L $ , which is a bottleneck in massive deployment scenarios. In this work, we show that such limitation can be overcome when the number of base station antennas $M$ is sufficiently large. More specifically, we prove that with a coherence block of dimension $L$ and a number of antennas $M$ such that $K_{a}/M = o(1)$ , one can identify $K_{a} = O\left({L^{2}/\log ^{2}\left({\frac {K_{\text {tot}}}{K_{a}}}\right)}\right)$ active users, which is much larger than the previously known bounds. We also provide two algorithms. One is based on Non-Negative Least-Squares, for which the above scaling result can be rigorously proved. The other consists of a low-complexity iterative componentwise minimization of the likelihood function of the underlying problem. While for this algorithm a rigorous proof cannot be given, we analyze a constrained version of the Maximum Likelihood (ML) problem (a combinatorial optimization with exponential complexity) and find the same fundamental scaling law for the number of identifiable users. Therefore, we conjecture that the low-complexity (approximated) ML algorithm also achieves the same scaling law and we demonstrate its performance by simulation. We also compare the discussed methods with the (Bayesian) MMV-AMP algorithm, recently proposed for the same setting, and show superior performance and better numerical stability. Finally, we use the discussed approximated ML algorithm as the inner decoder in a concatenated coding scheme for unsourced random access , a grant-free uncoordinated multiple access scheme where all users make use of the same codebook, and the receiver must produce the list of transmitted messages, irrespectively of the identity of the transmitters. We show that reliable communication is possible at any $E_{b}/N_{0}$ provided that a sufficiently large number of base station antennas is used, and that a sum spectral efficiency in the order of $\mathcal {O}(L\log (L))$ is achievable.

The Quantum Wasserstein Distance of Order 1

Abstract: We propose a generalization of the Wasserstein distance of order 1 to the quantum states of $n$ qudits. The proposal recovers the Hamming distance for the vectors of the canonical basis, and more generally the classical Wasserstein distance for quantum states diagonal in the canonical basis. The proposed distance is invariant with respect to permutations of the qudits and unitary operations acting on one qudit and is additive with respect to the tensor product. Our main result is a continuity bound for the von Neumann entropy with respect to the proposed distance, which significantly strengthens the best continuity bound with respect to the trace distance. We also propose a generalization of the Lipschitz constant to quantum observables. The notion of quantum Lipschitz constant allows us to compute the proposed distance with a semidefinite program. We prove a quantum version of Marton’s transportation inequality and a quantum Gaussian concentration inequality for the spectrum of quantum Lipschitz observables. Moreover, we derive bounds on the contraction coefficients of shallow quantum circuits and of the tensor product of one-qudit quantum channels with respect to the proposed distance. We discuss other possible applications in quantum machine learning, quantum Shannon theory, and quantum many-body systems.

Age of Information in G/G/1/1 Systems: Age Expressions, Bounds, Special Cases, and Optimization

Abstract: We consider the average age of information in G/G/1/1 systems under two service discipline models. In the first model, if a new update arrives when the service is busy, it is blocked; in the second model, a new update preempts the current update in service. For the blocking model, we first derive an exact age expression for G/G/1/1 systems. Then, using the age expression for G/G/1/1 systems, we calculate average age expressions for special cases, i.e., M/G/1/1 and G/M/1/1 systems. We observe that deterministic interarrivals minimize the average age of G/M/1/1 systems for a given mean interarrival time. Next, for the preemption in service model, we first derive an exact average age expression for G/G/1/1 systems. Then, similar to blocking discipline, using the age expression for G/G/1/1 systems, we calculate average age expressions for special cases, i.e., M/G/1/1 and G/M/1/1 systems. Average age for G/M/1/1 can be written as a summation of two terms, the first of which depends only on the first and second moments of interarrival times and the second of which depends only on the service rate. In other words, interarrival and service times are decoupled. We prove that deterministic interarrivals are optimum for G/M/1/1 systems for a given mean interarrival time. On the other hand, we observe for non-exponential service times that the optimal distribution of interarrival times depends on the relative values of the mean interarrival time and the mean service time. Finally, we propose a simple to calculate upper bound to the average age for the preemption in service discipline.

Deep Neural Network Approximation Theory

Abstract: This paper develops fundamental limits of deep neural network learning by characterizing what is possible if no constraints are imposed on the learning algorithm and on the amount of training data. Concretely, we consider Kolmogorov-optimal approximation through deep neural networks with the guiding theme being a relation between the complexity of the function (class) to be approximated and the complexity of the approximating network in terms of connectivity and memory requirements for storing the network topology and the associated quantized weights. The theory we develop establishes that deep networks are Kolmogorov-optimal approximants for markedly different function classes, such as unit balls in Besov spaces and modulation spaces. In addition, deep networks provide exponential approximation accuracy—i.e., the approximation error decays exponentially in the number of nonzero weights in the network—of the multiplication operation, polynomials, sinusoidal functions, and certain smooth functions. Moreover, this holds true even for one-dimensional oscillatory textures and the Weierstrass function—a fractal function, neither of which has previously known methods achieving exponential approximation accuracy. We also show that in the approximation of sufficiently smooth functions finite-width deep networks require strictly smaller connectivity than finite-depth wide networks.

Investigations on c -(Almost) Perfect Nonlinear Functions

Abstract: In a prior paper (Ellingsen et al. , 2020), two of us, along with P. Ellingsen, P. Felke, and A. Tkachenko, defined a new (output) multiplicative differential and the corresponding $c$ -differential uniformity, which has the potential of extending differential cryptanalysis. Here, we continue the work by looking at some APN functions through the mentioned concept and showing that their $c$ -differential uniformity increases significantly in some cases.

An Overview of Capacity Results for Synchronization Channels

Abstract: Synchronization channels, such as the well-known deletion channel, are surprisingly harder to analyze than memoryless channels, and they are a source of many fundamental problems in information theory and theoretical computer science. One of the most basic open problems regarding synchronization channels is the derivation of an exact expression for their capacity. Unfortunately, most of the classic information-theoretic techniques at our disposal fail spectacularly when applied to synchronization channels. Therefore, new approaches must be considered to tackle this problem. This survey gives an account of the great effort made over the past few decades to better understand the (broadly defined) capacity of synchronization channels, including both the main results and the novel techniques underlying them. Besides the usual notion of channel capacity, we also discuss the zero-error capacity of adversarial synchronization channels.

Reed–Muller Codes: Theory and Algorithms

Abstract: Reed-Muller (RM) codes are among the oldest, simplest and perhaps most ubiquitous family of codes They are used in many areas of coding theory in both electrical engineering and computer science Yet, many of their important properties are still under investigation This paper covers some of the recent developments regarding the weight enumerator and the capacity-achieving properties of RM codes, as well as some of the algorithmic developments In particular, the paper discusses the recent connections established between RM codes, thresholds of Boolean functions, polarization theory, hypercontractivity, and the techniques of approximating low weight codewords using lower degree polynomials (when codewords are viewed as evaluation vectors of degree $r$ polynomials in $m$ variables) It then overviews some of the algorithms for decoding RM codes It covers both algorithms with provable performance guarantees for every block length, as well as algorithms with state-of-the-art performances in practical regimes, which do not perform as well for large block length Finally, the paper concludes with a few open problems

Secure Coded Multi-Party Computation for Massive Matrix Operations

Abstract: In this article, we consider a secure multi-party computation problem (MPC), where the goal is to offload the computation of an arbitrary polynomial function of some massive private matrices (inputs) to a cluster of workers. The workers are not reliable. Some of them may collude to gain information about the input data (semi-honest workers). The system is initialized by sharing a (randomized) function of each input matrix to each server. Since the input matrices are massive, each share’s size is assumed to be at most $1/k$ fraction of the input matrix, for some $k \in \mathbb {N}$ . The objective is to minimize the number of workers needed to perform the computation task correctly, such that even if an arbitrary subset of $t-1$ workers, for some $t \in \mathbb {N}$ , collude, they cannot gain any information about the input matrices. We propose a sharing scheme, called polynomial sharing , and show that it admits basic operations such as adding and multiplication of matrices and transposing a matrix. By concatenating the procedures for basic operations, we show that any polynomial function of the input matrices can be calculated, subject to the problem constraints. We show that the proposed scheme can offer order-wise gain in terms of the number of workers needed, compared to the approaches formed by the concatenation of job splitting and conventional MPC approaches.

Levenshtein Distance, Sequence Comparison and Biological Database Search

Abstract: Levenshtein edit distance has played a central role—both past and present—in sequence alignment in particular and biological database similarity search in general. We start our review with a history of dynamic programming algorithms for computing Levenshtein distance and sequence alignments. Following, we describe how those algorithms led to heuristics employed in the most widely used software in bioinformatics, BLAST, a program to search DNA and protein databases for evolutionarily relevant similarities. More recently, the advent of modern genomic sequencing and the volume of data it generates has resulted in a return to the problem of local alignment. We conclude with how the mathematical formulation of Levenshtein distance as a metric made possible additional optimizations to similarity search in biological contexts. These modern optimizations are built around the low metric entropy and fractional dimensionality of biological databases, enabling orders of magnitude acceleration of biological similarity search.

An Infinite Family of Linear Codes Supporting 4-Designs

Abstract: The question as to whether there exists an infinite family of near MDS codes holding an infinite family of $t$ -designs for $t\geq 2$ was answered in the recent paper [Infinite families of near MDS codes holding $t$ -designs, IEEE Trans. Inf. Theory 66(9) (2020)], where an infinite family of near MDS codes holding an infinite family of 3-designs and an infinite family of near MDS codes holding an infinite family of 2-designs were presented, but no infinite family of linear codes holding an infinite family of 4-designs was presented. Hence, the question as to whether there is an infinite family of linear codes holding an infinite family of 4-designs remains open for 71 years. This paper settles this long-standing problem by presenting an infinite family of BCH codes of length $2^{2m+1}+1$ over ${\mathrm {GF}}(2^{2m+1})$ holding an infinite family of 4- $(2^{2m+1}+1, 6, 2^{2m}-4)$ designs. This paper also provides another solution to the first question, as some of the BCH codes presented in this paper are also near MDS. Moreover, an infinite family of linear codes holding the spherical geometry design $S(3, 5, 4^{m}+1)$ is presented. The new direction of searching for $t$ -designs with elementary symmetric polynomials will be further advanced.

Geometric Lower Bounds for Distributed Parameter Estimation Under Communication Constraints

Abstract: We consider parameter estimation in distributed networks, where each sensor in the network observes an independent sample from an underlying distribution and has $k$ bits to communicate its sample to a centralized processor which computes an estimate of a desired parameter. We develop lower bounds for the minimax risk of estimating the underlying parameter for a large class of losses and distributions. Our results show that under mild regularity conditions, the communication constraint reduces the effective sample size by a factor of $d$ when $k$ is small, where $d$ is the dimension of the estimated parameter. Furthermore, this penalty reduces at most exponentially with increasing $k$ , which is the case for some models, e.g., estimating high-dimensional distributions. For other models however, we show that the sample size reduction is re-mediated only linearly with increasing $k$ , e.g. when some sub-Gaussian structure is available. We apply our results to the distributed setting with product Bernoulli model, multinomial model, Gaussian location models, and logistic regression which recover or strengthen existing results. Our approach significantly deviates from existing approaches for developing information-theoretic lower bounds for communication-efficient estimation. We circumvent the need for strong data processing inequalities used in prior work and develop a geometric approach which builds on a new representation of the communication constraint. This approach allows us to strengthen and generalize existing results with simpler and more transparent proofs.

The Role of Coded Side Information in Single-Server Private Information Retrieval

Abstract: We study the role of coded side information in single-server Private Information Retrieval (PIR). An instance of the single-server PIR problem includes a server that stores a database of $K$ independently and uniformly distributed messages, and a user who wants to retrieve one of these messages from the server. We consider settings in which the user initially has access to a coded side information which includes a linear combination of a subset of $M$ messages in the database. We assume that the identities of the $M$ messages that form the support set of the coded side information as well as the coding coefficients are initially unknown to the server. We consider two different models, depending on whether the support set of the coded side information includes the requested message or not. We also consider the following two privacy requirements: (i) the identities of both the demand and the support set of the coded side information need to be protected, or (ii) only the identity of the demand needs to be protected. For each model and for each of the privacy requirements, we consider the problem of designing a protocol for generating the user’s query and the server’s answer that enables the user to decode the message they need while satisfying the privacy requirement. We characterize the (scalar-linear) capacity of each setting, defined as the ratio of the number of information bits in a message to the minimum number of information bits downloaded from the server over all (scalar-linear) protocols that satisfy the privacy condition. Our converse proofs rely on new information-theoretic arguments—tailored to the setting of single-server PIR and different from the commonly-used techniques in multi-server PIR settings. We also present novel capacity-achieving scalar-linear protocols for each of the settings being considered.

Asymmetric Leaky Private Information Retrieval

Abstract: Information-theoretic formulations of the private information retrieval (PIR) problem have been investigated under a variety of scenarios. Symmetric private information retrieval (SPIR) is a variant where a user is able to privately retrieve one out of $K$ messages from $N$ non-colluding replicated databases without learning anything about the remaining $K-1$ messages. However, the goal of perfect privacy can be too taxing for certain applications. In this paper, we investigate if the information-theoretic capacity of SPIR (equivalently, the inverse of the minimum download cost) can be increased by relaxing both user and DB privacy definitions. Such relaxation is relevant in applications where privacy can be traded for communication efficiency. We introduce and investigate the Asymmetric Leaky PIR (AL-PIR) model with different privacy leakage budgets in each direction. For user privacy leakage, we bound the probability ratios between all possible realizations of DB queries by a function of a non-negative constant $\epsilon $ . For DB privacy, we bound the mutual information between the undesired messages, the queries, and the answers, by a function of a non-negative constant $\delta $ . We propose a general AL-PIR scheme that achieves an upper bound on the optimal download cost for arbitrary $\epsilon $ and $\delta $ . We show that the optimal download cost of AL-PIR is upper-bounded as $D^{*}(\epsilon,\delta)\leq 1+\frac {1}{N-1}-\frac {\delta e^{\epsilon }}{N^{K-1}-1}$ . Second, we obtain an information-theoretic lower bound on the download cost as $D^{*}(\epsilon,\delta)\geq 1+\frac {1}{Ne^{\epsilon }-1}-\frac {\delta }{(Ne^{\epsilon })^{K-1}-1}$ . The gap analysis between the two bounds shows that our AL-PIR scheme is optimal when $\epsilon =0$ , i.e., under perfect user privacy and it is optimal within a maximum multiplicative gap of $\frac {N-e^{-\epsilon }}{N-1}$ for any $\epsilon >0$ and $\delta >0$ .

Improved Bounds and Singleton-Optimal Constructions of Locally Repairable Codes With Minimum Distance 5 and 6

Abstract: Repair locality has been an important metric in a distributed storage system (DSS). Erasure codes with small locality are more popular in a DSS, which means fewer available nodes participating in the repair process of failed nodes. Locally repairable codes (LRCs) as a new coding scheme have given more rise to the system performance and attracted a lot of interest in the theoretical research in coding theory. The particular concern among the research problems is the bounds and optimal constructions of LRCs. The problem of optimal constructions of LRCs includes the most important case of Singleton-optimal LRCs whose minimum distance achieves the Singleton-like bound, which is the core consideration in this paper. In this work, we first of all derive an improved and general upper bound on the code length of Singleton-optimal LRCs with minimum distance $d=5, 6$ , some known constructions are shown to exactly achieve our new bound, which verifies its tightness. For locality $r=2$ and distance $d=6$ , we construct three new Singleton-optimal LRCs whose code length $n=3(q+1)$ , $n=3(q+\sqrt {q}+1)$ and $n=3(2q-4)$ , respectively. Moreover, we obtain a complete characterization for Singleton-optimal LRCs with $r=2$ and $d=6$ . Such characterization has established an important connection between the existence of Singleton-optimal LRCs and that of a special subset of lines of finite projective plane $PG(2, q)$ , thus provides a methodology for constructing LRCs with longer length based on any advance on finite projective plane $PG(2, q)$ . In the end, we employ the well-known line-point incidence matrix and Johnson bounds for constant weight codes to derive tighter upper bounds on the code length. These new bounds further help us to prove that some of the previous Singleton-optimal constructions or their extensions achieve the longest possible code length for $q=3, 4, 5, 7$ . It’s worth noting that all of our Singleton-optimal constructions possess small locality $r=2$ , which are attractive in a DSS.

Balanced Product Quantum Codes

Abstract: This work provides the first explicit and non-random family of $[[N,K,D]]$ LDPC quantum codes which encode $K \in \Theta \left({N^{\frac {4}{5}}}\right)$ logical qubits with distance $D \in \Omega \left({N^{\frac {3}{5}}}\right)$ . The family is constructed by amalgamating classical codes and Ramanujan graphs via an operation called balanced product . Recently, Hastings–Haah–O’Donnell and Panteleev–Kalachev were the first to show that there exist families of LDPC quantum codes which break the $\mathrm {polylog}(N)\sqrt {N}$ distance barrier. However, their constructions are based on probabilistic arguments which only guarantee the code parameters with high probability whereas our bounds hold unconditionally. Further, balanced products allow for non-abelian twisting of the check matrices, leading to a construction of LDPC quantum codes that can be shown to have $K\in \Theta (N)$ and that we conjecture to have linear distance $D\in \Theta (N)$ .

Synchronization Strings and Codes for Insertions and Deletions—A Survey

Abstract: Already in the 1960s, Levenshtein and others studied error-correcting codes that protect against synchronization errors, such as symbol insertions and deletions However, despite significant efforts, progress on designing such codes has been lagging until recently, particularly compared to the detailed understanding of error-correcting codes for symbol substitution or erasure errors This paper surveys the recent progress in designing efficient error-correcting codes over finite alphabets that can correct a constant fraction of worst-case insertions and deletions Most state-of-the-art results for such codes rely on synchronization strings, simple yet powerful pseudo-random objects that have proven to be very effective solutions for coping with synchronization errors in various settings This survey also includes an overview of what is known about synchronization strings and discusses communication settings related to error-correcting codes in which synchronization strings have been applied

Optimal Sampling and Scheduling for Timely Status Updates in Multi-Source Networks

Abstract: We consider a joint sampling and scheduling problem for optimizing data freshness in multi-source systems. Data freshness is measured by a non-decreasing penalty function of age of information , where all sources have the same age-penalty function. Sources take turns to generate update packets, and forward them to their destinations one-by-one through a shared channel with random delay. There is a scheduler, that chooses the update order of the sources, and a sampler, that determines when a source should generate a new packet in its turn. We aim to find the optimal scheduler-sampler pairs that minimize the total-average age-penalty at delivery times (Ta-APD) and the total-average age-penalty (Ta-AP). We prove that the Maximum Age First (MAF) scheduler and the zero-wait sampler are jointly optimal for minimizing the Ta-APD. Meanwhile, the MAF scheduler and a relative value iteration with reduced complexity (RVI-RC) sampler are jointly optimal for minimizing the Ta-AP. The RVI-RC sampler is based on a relative value iteration algorithm whose complexity is reduced by exploiting a threshold property in the optimal sampler. Finally, a low-complexity threshold-type sampler is devised via an approximate analysis of Bellman’s equation. This threshold-type sampler reduces to a simple water-filling sampler for a linear age-penalty function.

DNA-Based Storage: Models and Fundamental Limits

Abstract: Due to its longevity and enormous information density, DNA is an attractive medium for archival storage. In this work, we study the fundamental limits and trade-offs of DNA-based storage systems by introducing a new channel model, which we call the noisy shuffling-sampling channel. Motivated by current technological constraints on DNA synthesis and sequencing, this model captures three key distinctive aspects of DNA storage systems: (1) the data is written onto many short DNA molecules; (2) the molecules are corrupted by noise during synthesis and sequencing and (3) the data is read by randomly sampling from the DNA pool. We provide capacity results for this channel under specific noise and sampling assumptions and show that, in many scenarios, a simple index-based coding scheme is optimal.

On Optimal k -Deletion Correcting Codes

Abstract: Levenshtein introduced the problem of constructing k -deletion correcting codes in 1966, proved that the optimal redundancy of those codes is $ {O}(k~\log ~{N})$ for constant k, and proposed an optimal redundancy single-deletion correcting code (using the so-called VT construction). However, the problem of constructing optimal redundancy k -deletion correcting codes remained open. Our key contribution is a major step towards a complete solution to this longstanding open problem for constant k . We present a k -deletion correcting code that has redundancy $8 {k}\log~{N} + {o}(\log~{N})$ when $ {k}= {o}(\sqrt {\log \log~{N}})$ and encoding/decoding algorithms of complexity $ {O}({N}^{2 {k}+1})$ .

Instance-Dependent ℓ ∞ -Bounds for Policy Evaluation in Tabular Reinforcement Learning

Abstract: Markov reward processes (MRPs) are used to model stochastic phenomena arising in operations research, control engineering, robotics, and artificial intelligence, as well as communication and transportation networks. In many of these cases, such as in the policy evaluation problem encountered in reinforcement learning, the goal is to estimate the long-term value function of such a process without access to the underlying population transition and reward functions. Working with samples generated under the synchronous model, we study the problem of estimating the value function of an infinite-horizon discounted MRP with finite state space in the $\ell _{\infty }$ -norm. We analyze both the standard plug-in approach to this problem and a more robust variant, and establish non-asymptotic bounds that depend on the (unknown) problem instance, as well as data-dependent bounds that can be evaluated based on the observations of state-transitions and rewards. We show that these approaches are minimax-optimal up to constant factors over natural sub-classes of MRPs. Our analysis makes use of a leave-one-out decoupling argument tailored to the policy evaluation problem, one which may be of independent interest.

On Coded Caching With Private Demands

Abstract: Caching is an efficient way to reduce network traffic congestion during peak hours by storing some content at the user’s local cache memory without knowledge of later demands. For the shared-link caching model, Maddah-Ali and Niesen (MAN) proposed a two-phase ( placement and delivery ) coded caching strategy, which is order optimal within a constant factor. However, in the MAN coded caching scheme, each user can obtain the information about the demands of other users, i.e., the MAN coded caching scheme is inherently prone to tampering and spying the activity/demands of other users. In this paper, we formulate an information-theoretic shared-link caching model with private demands, where there are ${\mathsf K}$ cache-aided users (which can cache up to ${\mathsf M}$ files) connected to a central server with access to ${\mathsf N}$ files. Each user requests ${\mathsf L}$ files. Our objective is to design a two-phase private caching scheme with minimum load while preserving the information-theoretic privacy of the demands of each user with respect to other users. A trivial solution is the uncoded caching scheme which lets each user recover all the ${\mathsf N}$ files, referred to as baseline scheme . For this problem we propose two novel schemes which achieve the information-theoretic privacy of the users’ demands while also achieving a non-trivial caching gain over the baseline scheme. The general underlying idea is to satisfy the users’ requests by generating a set of coded multicast messages that is symmetric with respect to the library files, such that for each user k, the mutual information between these messages and the demands of all other users given the cache content and the demand of user k is zero. In the first scheme, referred to as virtual-user scheme, we introduce a number of virtual users such that each ${\mathsf L}$ -subset of files is demanded by ${\mathsf K}$ real or virtual (effective) users and use the MAN delivery to generate multicast messages. From the viewpoint of each user, the set of multicast messages is symmetric over all files even if each single multicast message is not. This scheme incurs in an extremely large sub-packetization. Then, we propose a second scheme, referred to as MDS-based scheme, based on a novel MDS-coded cache placement. In this case, we generate multicast messages where each multicast message contains one MDS-coded symbol from each file in the library and thus is again symmetric over all the files from the viewpoint of each user. The sub-packetization level of the MDS-based scheme is exponentially smaller than that needed by the virtual-user scheme. Compared with the existing shared-link coded caching converse bounds without privacy, the virtual-user scheme is proved to be order optimal with a constant factor when ${\mathsf N}\leq {\mathsf L} {\mathsf K} $ , or when ${\mathsf N}\geq {\mathsf L} {\mathsf K} $ and ${\mathsf M}\geq {\mathsf N}/ {\mathsf K}$ . In addition, when ${\mathsf M}\geq {\mathsf N}/2$ , both of the virtual-user scheme and the MDS-based scheme are order optimal within a factor of 2.

Correcting a Single Indel/Edit for DNA-Based Data Storage: Linear-Time Encoders and Order-Optimality

Abstract: An indel refers to a single insertion or deletion, while an edit refers to a single insertion, deletion or substitution. In this article, we investigate codes that correct either a single indel or a single edit and provide linear-time algorithms that encode binary messages into these codes of length n. Over the quaternary alphabet, we provide two linear-time encoders. One corrects a single edit with $\lceil {\log \text {n}}\rceil+\text {O}(\log \log \text {n})$ redundancy bits, while the other corrects a single indel with $\lceil {\log \text {n}}\rceil+2$ redundant bits. These two encoders are order-optimal . The former encoder is the first known order-optimal encoder that corrects a single edit, while the latter encoder (that corrects a single indel) reduces the redundancy of the best known encoder of Tenengolts (1984) by at least four bits. Over the DNA alphabet, we impose an additional constraint: the $\mathtt {GC}$ -balanced constraint and require that exactly half of the symbols of any DNA codeword to be either $\mathtt {C}$ or $\mathtt {G}$ . In particular, via a modification of Knuth’s balancing technique, we provide a linear-time map that translates binary messages into $\mathtt {GC}$ -balanced codewords and the resulting codebook is able to correct a single indel or a single edit. These are the first known constructions of $\mathtt {GC}$ -balanced codes that correct a single indel or a single edit.

Capacity Optimality of AMP in Coded Systems

Abstract: This paper studies a large random matrix system (LRMS) model involving an arbitrary signal distribution and forward error control (FEC) coding. We establish an area property based on the approximate message passing (AMP) algorithm. Under the assumption that the state evolution for AMP is correct for the coded system, the achievable rate of AMP is analyzed. We prove that AMP achieves the constrained capacity of the LRMS with an arbitrary signal distribution provided that a matching condition is satisfied. As a byproduct, we provide an alternative derivation for the constraint capacity of an LRMS using a proved property of AMP. We discuss realization techniques for the matching principle of binary signaling using irregular low-density parity-check (LDPC) codes and provide related numerical results. We show that the optimized codes demonstrate significantly better performance over un-matched ones under AMP. For quadrature phase shift keying (QPSK) modulation, bit error rate (BER) performance within 1 dB from the constrained capacity limit is observed.

Fundamental Properties of Sum-Rank-Metric Codes

Abstract: This paper investigates the theory of sum-rank-metric codes for which the individual matrix blocks may have different sizes. Various bounds on the cardinality of a code are derived, along with their asymptotic extensions. The duality theory of sum-rank-metric codes is also explored, showing that MSRD codes (the sum-rank analogue of MDS codes) dualize to MSRD codes only if all matrix blocks have the same number of columns. In the latter case, duality considerations lead to an upper bound on the number of blocks for MSRD codes. The paper also contains various constructions of sum-rank-metric codes for variable block sizes, illustrating the possible behaviours of these objects with respect to bounds, existence, and duality properties.

SPARCs for Unsourced Random Access

Abstract: Unsourced random-access (U-RA) is a type of grant-free random access with a virtually unlimited number of users, of which only a certain number $K_{a}$ are active on the same time slot. Users employ exactly the same codebook, and the task of the receiver is to decode the list of transmitted messages. We present a concatenated coding construction for U-RA on the AWGN channel, in which a sparse regression code (SPARC) is used as an inner code to create an effective outer OR-channel. Then an outer code is used to resolve the multiple-access interference in the OR-MAC. We propose a modified version of the approximate message passing (AMP) algorithm as an inner decoder and give a precise asymptotic analysis of the error probabilities of the AMP decoder and of a hypothetical optimal inner MAP decoder. This analysis shows that the concatenated construction under optimal decoding can achieve a vanishing per-user error probability in the limit of large blocklength and a large number of active users at sum-rates up to the symmetric Shannon capacity, i.e. as long as $K_{a}R . This extends previous point-to-point optimality results about SPARCs to the unsourced multiuser scenario. Furthermore, we give an optimization algorithm to find the power allocation for the inner SPARC code that minimizes the ${\mathsf {SNR}}$ required to achieve a given target per-user error probability with the AMP decoder.

Distributed Online Linear Regressions

Abstract: We study online linear regression problems in a distributed setting, where the data is spread over a network. In each round, each network node proposes a linear predictor, with the objective of fitting the network-wide data. It then updates its predictor for the next round according to the received local feedback and information received from neighboring nodes. The predictions made at a given node are assessed through the notion of regret, defined as the difference between their cumulative network-wide square errors and those of the best off-line network-wide linear predictor. Various scenarios are investigated, depending on the nature of the local feedback (full information or bandit feedback), on the set of available predictors (the decision set), and the way data is generated (by an oblivious or adaptive adversary). We propose simple and natural distributed regression algorithms, involving, at each node and in each round, a local gradient descent step and a communication and averaging step where nodes aim at aligning their predictors to those of their neighbors. We establish regret upper bounds typically in ${\mathcal{ O}}(T^{3/4})$ when the decision set is unbounded and in ${\mathcal{ O}}(\sqrt {T})$ in case of bounded decision set.

Sublinear-Time Non-Adaptive Group Testing With O ( k log n ) Tests via Bit-Mixing Coding

Abstract: The group testing problem consists of determining a small set of defective items from a larger set of items based on tests on groups of items, and is relevant in applications such as medical testing, communication protocols, pattern matching, and many more. While rigorous group testing algorithms have long been known with runtime at least linear in the number of items, a recent line of works has sought to reduce the runtime to ${\mathrm{ poly}}({k} \log {n})$ , where n is the number of items and k is the number of defectives. In this paper, we present such an algorithm for non-adaptive group testing termed bit mixing coding (BMC), which builds on techniques that encode item indices in the test matrix, while incorporating novel ideas based on erasure-correction coding. We show that BMC achieves asymptotically vanishing error probability with ${O}({k} \log {n})$ tests and ${O}({k}^{2} \cdot \log {k} \cdot \log {n})$ runtime, in the limit as ${n} \to \infty $ (with k having an arbitrary dependence on n). This closes an open problem of simultaneously achieving ${\mathrm{ poly}}({k} \log {n})$ decoding time using ${O}({k} \log {n})$ tests without any assumptions on k. In addition, we show that the same scaling laws can be attained in a commonly-considered noisy setting, in which each test outcome is flipped with constant probability.

Quantum LDPC Codes with Almost Linear Minimum Distance

Abstract: We give a construction of quantum LDPC codes of dimension Θ(log N) and distance Θ(N/log N) as the code length N → ∞. Using a product of chain complexes this construction also provides a family of quantum LDPC codes of distance Ω(N1-α/2/log N) and dimension Ω(Nα log N), where 0 ≤ α < 1. We also introduce and study a new operation called lifted product, which naturally generalizes the product operations for quantum codes and chain complexes. Moreover, as a simple byproduct of our results on quantum codes, we obtain a new result on classical codes. We show that for any fixed R < 1 there exists an asymptotically good family of classical quasi-cyclic LDPC codes of rate at least R with, in some sense, optimal circulant size Ω(N/log N) as the code length N → ∞.

CodedSketch: A Coding Scheme for Distributed Computation of Approximated Matrix Multiplication

Abstract: In this paper, we propose CodedSketch , as a distributed straggler-resistant scheme to compute an approximation of the multiplication of two massive matrices. The objective is to reduce the recovery threshold , defined as the total number of worker nodes that the master node needs to wait for to be able to recover the final result. To exploit the fact that only an approximated result is required, in reducing the recovery threshold, some sorts of pre-compression are required. However, compression inherently involves some randomness that would lose the structure of the matrices. On the other hand, considering the structure of the matrices is crucial to reduce the recovery threshold. In CodedSketch, we use count–sketch, as a hash-based compression scheme, on the rows of the first and columns of the second matrix, and a structured polynomial code on the columns of the first and rows of the second matrix. This arrangement allows us to exploit the gain of both in reducing the recovery threshold. To increase the accuracy of computation, multiple independent count–sketches are needed. This independency allows us to theoretically characterize the accuracy of the result and establish the recovery threshold achieved by the proposed scheme. To guarantee the independency of resulting count–sketches in the output, while keeping its cost on the recovery threshold minimum, we use another layer of structured codes. The proposed scheme provides an upper-bound on the recovery threshold as a function of the required accuracy of computation and the probability that the required accuracy can be violated. In addition, it provides an upper-bound on the recovery threshold for the case that the result of the multiplication is sparse, and the exact result is required.

New LCD MDS Codes of Non-Reed-Solomon Type

Abstract: Both linear complementary dual (LCD) codes and maximum distance separable (MDS) codes have good algebraic structures, and they have interesting practical applications such as communication systems, data storage, quantum codes, and so on. So far, most of LCD MDS codes have been constructed by employing generalized Reed-Solomon codes. In this paper we construct some classes of new Euclidean LCD MDS codes and Hermitian LCD MDS codes which are not monomially equivalent to Reed-Solomon codes, called LCD MDS codes of non-Reed-Solomon type. Our method is based on the constructions of Beelen et al. (2017) and Roth and Lempel (1989). To the best of our knowledge, this is the first paper on the construction of LCD MDS codes of non-Reed-Solomon type; any LCD MDS code of non-Reed-Solomon type constructed by our method is not monomially equivalent to any LCD code constructed by the method of Carlet et al. (2018).