国際会議開催報告:2013 IEEE International Symposium on Information Theory

An $(n,k,l)$ maximum distance separable (MDS) array code of length $n$ , dimension $k=n-r$ , and sub-packetization $l$ is formed of $l\times n$ matrices over a finite field $F$ , with every column of the matrix stored on a separate node in the distributed storage system and viewed as a coordinate of the codeword. Repair of a failed node (recovery of one erased column) can be performed by accessing a set of $d\le n-1$ surviving (helper) nodes. The code is said to have the optimal access property if the amount of data accessed at each of the helper nodes meets a lower bound on this quantity. For optimal-access MDS codes with $d=n-1$ , the sub-packetization $l$ satisfies the bound $l\ge r^{(k-1)/r}$ . In our previous work (IEEE Trans. Inf. Theory, vol. 63, no. 4, 2017), for any $n$ and $r$ , we presented an explicit construction of optimal-access MDS codes with sub-packetization $l=r^{n-1}$ . In this paper, we take up the question of reducing the sub-packetization value $l$ to make it to approach the lower bound. We construct an explicit family of optimal-access codes with $l=r^{\lceil n/r\rceil }$ , which differs from the optimal value by at most a factor of $r^{2}$ . These codes can be constructed over any finite field $F$ as long as $|F|\ge r\lceil n/r\rceil $ , and afford low-complexity encoding and decoding procedures. We also define a version of the repair problem that bridges the context of regenerating codes and codes with locality constraints (LRC codes), which we call group repair with optimal access . In this variation, we assume that the set of $n=sm$ nodes is partitioned into $m$ repair groups of size $s$ , and require that the amount of accessed data for repair is the smallest possible whenever the $d=s+k-1$ helper nodes include all the other $s-1$ nodes from the same group as the failed node. For this problem, we construct a family of codes with the group optimal access property. These codes can be constructed over any field $F$ of size $|F|\ge n$ , and also afford low-complexity encoding and decoding procedures.

Explicit Constructions of Optimal-Access MDS Codes With Nearly Optimal Sub-Packetization

We study the performance of Reed–Solomon (RS) codes for the exact repair problem in distributed storage. Our main result is that, in some parameter regimes, Reed–Solomon codes are optimal regenerating codes, among maximum distance separable (MDS) codes with linear repair schemes. Moreover, we give a characterization of MDS codes with linear repair schemes, which holds in any parameter regime, and which can be used to give non-trivial repair schemes for RS codes in other settings. More precisely, we show that for $k$ -dimensional RS codes whose evaluation points are a finite field of size $n$ , there are exact repair schemes with bandwidth $(n-1)\log ((n-1)/(n-k))$ bits, and that this is optimal for any MDS code with a linear repair scheme. In contrast, the naive (commonly implemented) repair algorithm for this RS code has bandwidth $k\log (n)$ bits. When the entire field is used as evaluation points, the number of nodes $n$ is much larger than the number of bits per node (which is $O(\log (n))$ ), and so this result holds only when the degree of sub-packetization is small. However, our method applies in any parameter regime, and to illustrate this for high levels of sub-packetization, we give an improved repair scheme for a specific (14,10)-RS code used in the facebook hadoop analytics cluster.

Repairing Reed-Solomon Codes

Regenerating codes for distributed storage have attracted much research interest in the past decade. Such codes trade the bandwidth needed to repair a failed node with the overall amount of data stored in the network. Minimum storage regenerating (MSR) codes are an important class of optimal regenerating codes that minimize (first) the amount of data stored per node and (then) the repair bandwidth. Specifically, an $[n,k,d]$ - $(\alpha )$ MSR code $ \mathbb {C}$ over $ \smash {\mathbb {F}_{\!q}}$ stores a file $ {\mathcal{ F}}$ consisting of $\alpha k$ symbols over $ \smash {\mathbb {F}_{\!q}}$ among $n$ nodes, each storing $\alpha $ symbols, in such a way that: 1) the file $ {\mathcal{ F}}$ can be recovered by downloading the content of any $k$ of the $n$ nodes and 2) the content of any failed node can be reconstructed by accessing any $d$ of the remaining $n-1$ nodes and downloading $\alpha /(d{-}k{+}1)$ symbols from each of these nodes. In practice, the file $ {\mathcal{ F}}$ is typically available in uncoded form on some $k$ of the $n$ nodes, known as systematic nodes , and the defining node-repair condition above can be relaxed to requiring the optimal repair bandwidth for systematic nodes only . Such codes are called systematic–repair MSR codes . Unfortunately, finite– $\alpha $ constructions of $[n,k,d]$ MSR codes are known only for certain special cases: either low rate, namely $k/n \leqslant 0.5$ , or high repair connectivity, namely $d = n-1$ . Our main result in this paper is a finite– $\alpha $ construction of systematic-repair $[n,k,d]$ MSR codes for all possible values of parameters $n,k,d$ . We also introduce a generalized construction for $[n,k]$ MSR codes to achieve the optimal repair bandwidth for all values of $d$ simultaneously.

/pdf/minimum-storage-regenerating-codes-for-all-parameters-4f9yuocs14.pdf

Minimum Storage Regenerating Codes for All Parameters

This paper presents an explicit construction for an $((n,k,d=n-1), (\alpha,\beta))$ regenerating code over a field $\mathbb{F}_Q$ operating at the Minimum Storage Regeneration (MSR) point. The MSR code can be constructed to have rate $k/n$ as close to $1$ as desired, sub-packetization given by $r^{\frac{n}{r}}$, for $r=(n-k)$, field size no larger than $n$ and where all code symbols can be repaired with the same minimum data download. The construction modifies a prior construction by Sasidharan et. al. which required far larger field-size. A building block appearing in the construction is a scalar MDS code of block length $n$. The code has a simple layered structure with coupling across layers, that allows both node repair and data recovery to be carried out by making multiple calls to a decoder for the scalar MDS code. While this work was carried out independently, there is considerable overlap with a prior construction by Ye and Barg. 
It is shown here that essentially the same architecture can be employed to construct MSR codes using vector binary MDS codes as building blocks in place of scalar MDS codes. The advantage here is that computations can now be carried out over a field of smaller size potentially even over the binary field as we demonstrate in an example. Further, we show how the construction can be extended to handle the case of $d<(n-1)$ under a mild restriction on the choice of helper nodes.

An Explicit, Coupled-Layer Construction of a High-Rate MSR Code with Low Sub-Packetization Level, Small Field Size and All-Node Repair

In this paper, we study the notion of codes with hierarchical locality that is identified as another approach to local recovery from multiple erasures. The well-known class of codes with locality is said to possess hierarchical locality with a single level. In a code with two-level hierarchical locality, every symbol is protected by an inner-most local code, and another middle-level code of larger dimension containing the local code. We first consider codes with two levels of hierarchical locality, derive an upper bound on the minimum distance, and provide optimal code constructions of low field-size under certain parameter sets. Subsequently, we generalize both the bound and the constructions to hierarchical locality of arbitrary levels.

Codes with hierarchical locality

We present a high-rate (n, k, d = n − 1)-MSR code with a sub-packetization level that is polynomial in the dimension k of the code While polynomial sub-packetization level was achieved earlier for vector MDS codes that repair systematic nodes optimally, no such MSR code construction is known In the low-rate regime (i e, rates less than one-half), MSR code constructions with a linear sub-packetization level are available But in the high-rate regime (i e, rates greater than one-half), the known MSR code constructions required a sub-packetization level that is exponential in k In the present paper, we construct an MSR code for d = n − 1 with a fixed rate equation, achieveing a sub-packetization level α = O(kt) The code allows help-by-transfer repair, i e, no computations are needed at the helper nodes during repair of a failed node

A high-rate MSR code with polynomial sub-packetization level

We present a high-rate $(n,k,d=n-1)$-MSR code with a sub-packetization level that is polynomial in the dimension $k$ of the code. While polynomial sub-packetization level was achieved earlier for vector MDS codes that repair systematic nodes optimally, no such MSR code construction is known. In the low-rate regime (i. e., rates less than one-half), MSR code constructions with a linear sub-packetization level are available. But in the high-rate regime (i. e., rates greater than one-half), the known MSR code constructions required a sub-packetization level that is exponential in $k$. In the present paper, we construct an MSR code for $d=n-1$ with a fixed rate $R=\frac{t-1}{t}, \ t \geq 2,$ achieveing a sub-packetization level $\alpha = O(k^t)$. The code allows help-by-transfer repair, i. e., no computations are needed at the helper nodes during repair of a failed node.

A High-Rate MSR Code With Polynomial Sub-Packetization Level

Given the scale of today's distributed storage systems, the failure of an individual node is a common phenomenon. Various metrics have been proposed to measure the efficacy of the repair of a failed node, such as the amount of data download needed to repair (also known as the repair bandwidth), the amount of data accessed at the helper nodes, and the number of helper nodes contacted. Clearly, the amount of data accessed can never be smaller than the repair bandwidth. In the case of a help-by-transfer code, the amount of data accessed is equal to the repair bandwidth. It follows that a help-by-transfer code possessing optimal repair bandwidth is access optimal. The focus of the present paper is on help-by-transfer codes that employ minimum possible bandwidth to repair the systematic nodes and are thus access optimal for the repair of a systematic node. The zigzag construction by Tamo et al. in which both systematic and parity nodes are repaired is access optimal. But the sub-packetization level required is rk where r is the number of parities and k is the number of systematic nodes. To date, the best known achievable sub-packetization level for access-optimal codes is rk/r in a MISER-code-based construction by Cadambe et al. in which only the systematic nodes are repaired and where the location of symbols transmitted by a helper node depends only on the failed node and is the same for all helper nodes. Under this set-up, it turns out that this sub-packetization level cannot be improved upon. In the present paper, we present an alternate construction under the same setup, of an access-optimal code repairing systematic nodes, that is inspired by the zigzag code construction and that also achieves a sub-packetization level of rk/r.

An alternate construction of an access-optimal regenerating code with optimal sub-packetization level

This paper considers multiple unicast wireline noiseless networks where a single source wishes to transmit independent messages to a set of legitimate destinations. The primal goal is to characterize the secure capacity region, where the exchanged messages have to be secured from a passive external eavesdropper that has unbounded computational capabilities, but limited network presence. The secure capacity region for the case of two destinations is characterized and it is shown to be a function of only the min-cut capacities and the number of edges the eavesdropper wiretaps. A polynomial-time two-phase scheme is then designed for a general number of destinations and its achievable secure rate region is derived. It is shown that the secure capacity result for the two destinations case is not reversible, that is, by switching the role of the source and destinations and by reversing the directions of the edges, the secure capacity region changes.

https://par.nsf.gov/servlets/purl/10065996

Gaurav Kumar Agarwal

Papers

Codes with hierarchical locality

A high-rate MSR code with polynomial sub-packetization level

A High-Rate MSR Code With Polynomial Sub-Packetization Level

An alternate construction of an access-optimal regenerating code with optimal sub-packetization level

Secure Network Coding for Multiple Unicast: On the Case of Single Source