Results on algorithmic problems on strings that are given in a compressed form via straightline programs are surveyed. A straight-line program is a context-free grammar that generates exactly one string. In this way, exponential compression rates can be achieved. Among others, we study pattern matching for compressed strings, membership problems for compressed strings in various kinds of formal languages, and the problem of querying compressed strings. Applications in combinatorial group theory and computational topology and to the solution of word equations are discussed as well. Finally, extensions to compressed trees and pictures are considered.

https://www.informatik.uni-leipzig.de/~lohrey/12-SURVEY.pdf

Algorithmics on SLP-compressed strings: A survey

Grammar-based compression, where one replaces a long string by a small context-free grammar that generates the string, is a simple and powerful paradigm that captures (sometimes with slight reduction in efficiency) many of the popular compression schemes, including the Lempel--Ziv family, run-length encoding, byte-pair encoding, Sequitur, and Re-Pair. In this paper, we present a novel grammar representation that allows efficient random access to any character or substring without decompressing the string. Let $S$ be a string of length $N$ compressed into a context-free grammar $\mathcal{S}$ of size $n$. We present two representations of $\mathcal{S}$ achieving $O(\log N)$ random access time, and either $O(n\cdot\alpha_k(n))$ construction time and space on the pointer machine model, or $O(n)$ construction time and space on the RAM. Here, $\alpha_k(n)$ is the inverse of the $k$th row of Ackermann's function. Our representations also efficiently support decompression of any substring in $S$: we can decompres...

/pdf/random-access-to-grammar-compressed-strings-and-trees-1zo5yciojo.pdf

Random Access to Grammar-Compressed Strings and Trees

A well-known fact in the field of lossless text compression is that high-order entropy is a weak model when the input contains long repetitions. Motivated by this fact, decades of research have generated myriads of so-called dictionary compressors: algorithms able to reduce the text’s size by exploiting its repetitiveness. Lempel-Ziv 77 is one of the most successful and well-known tools of this kind, followed by straight-line programs, run-length Burrows-Wheeler transform, macro schemes, collage systems, and the compact directed acyclic word graph. In this paper, we show that these techniques are different solutions to the same, elegant, combinatorial problem: to find a small set of positions capturing all distinct text’s substrings. We call such a set a string attractor. We first show reductions between dictionary compressors and string attractors. This gives the approximation ratios of dictionary compressors with respect to the smallest string attractor and allows us to uncover new asymptotic relations between the output sizes of different dictionary compressors. We then show that the k-attractor problem — deciding whether a text has a size-t set of positions capturing all substrings of length at most k — is NP-complete for k≥ 3. This, in particular, includes the full string attractor problem. We provide several approximation techniques for the smallest k-attractor, show that the problem is APX-complete for constant k, and give strong inapproximability results. To conclude, we provide matching lower and upper bounds for the random access problem on string attractors. The upper bound is proved by showing a data structure supporting queries in optimal time. Our data structure is universal: by our reductions to string attractors, it supports random access on any dictionary-compression scheme. In particular, it matches the lower bound also on LZ77, straight-line programs, collage systems, and macro schemes, and therefore essentially closes (at once) the random access problem for all these compressors.

/pdf/at-the-roots-of-dictionary-compression-string-attractors-309dh9wc18.pdf

At the roots of dictionary compression: string attractors

To store and search genomic databases efficiently, researchers have recently started building self-indexes based on LZ77. As the name suggests, a self-index for a string supports both random access and pattern matching queries. In this paper we show how, given a string S [1..n] whose LZ77 parse consists of z phrases, we can store a self-index for S in $\mathcal{O}({z \log (n / z)})$ space such that later, first, given a position i and a length l, we can extract S [i..i + l − 1] in $\mathcal{O}({\ell + \log n})$ time; second, given a pattern P [1..m], we can list the occ occurrences of P in S in $\mathcal{O}({m \log m + occ \log \log n})$ time.

LZ77-Based Self-indexing with Faster Pattern Matching

We study the problem of estimating the maximum matching size in graphs whose edges are revealed in a streaming manner. We consider both insertion-only streams, which only contain edge insertions, and dynamic streams that allow both insertions and deletions of the edges, and present new upper and lower bound results for both cases.On the upper bound front, we show that an α-approximate estimate of the matching size can be computed in dynamic streams using O(n2/α4) space, and in insertion-only streams using O(n/α2)-space. These bounds respectively shave off a factor of α from the space necessary to compute an α-approximate matching (as opposed to only size), thus proving a non-trivial separation between approximate estimation and approximate computation of matchings in data streams.On the lower bound front, we prove that any α-approximation algorithm for estimating matching size in dynamic graph streams requires [EQUATION] bits of space, even if the underlying graph is both sparse and has arboricity bounded by O(α). We further improve our lower bound to Ω(n/α2) in the case of dense graphs. These results establish the first non-trivial streaming lower bounds for super-constant approximation of matching size.Furthermore, we present the first super-linear space lower bound for computing a (1 + e)-approximation of matching size even in insertion-only streams. In particular, we prove that a (1 + e)-approximation to matching size requires RS(n) · n1−O(e) space; here, RS(n) denotes the maximum number of edge-disjoint induced matchings of size Θ(n) in an n-vertex graph. It is a major open problem with far-reaching implications to determine the value of RS(n), and current results leave open the possibility that RS(n) may be as large as n/log n. Moreover, using the best known lower bounds for RS(n), our result already rules out any O(n · poly(log n/e))-space algorithm for (1 + e)-approximation of matchings. We also show how to avoid the dependency on the parameter RS(n) in proving lower bound for dynamic streams and present a near-optimal lower bound of n2−O(e) for (1 + e)-approximation in this model.Using a well-known connection between matching size and matrix rank, all our lower bounds also hold for the problem of estimating matrix rank. In particular our results imply a near-optimal n2−O(e) bit lower bound for (1 + e)-approximation of matrix ranks for dense matrices in dynamic streams, answering an open question of Li and Woodruff (STOC 2016).

On estimating maximum matching size in graph streams

In this paper we introduce a new technique for proving streaming lower bounds (and one-way communication lower bounds), by reductions from a problem called the Boolean Hidden Hypermatching problem (BHH). BHH is a generalization of the well-known Boolean Hidden Matching problem, which was used by Gavinsky et al. to prove an exponential separation between quantum communication complexity and one-way randomized communication complexity. We are the first to introduce BHH, and to prove a lower bound for it.The hardness of the BHH problem is inherently oneway: it is easy to solve BHH using logarithmic two-way communication, but it requires √n communication if Alice is only allowed to send messages to Bob, and not vice-versa. This one-wayness allows us to prove lower bounds, via reductions, for streaming problems and related communication problems whose hardness is also inherently one-way.By designing reductions from BHH, we prove lower bounds for the streaming complexity of approximating the sorting by reversal distance, of approximately counting the number of cycles in a 2-regular graph, and of other problems.For example, here is one lower bound that we prove, for a cycle-counting problem: Alice gets a perfect matching EA on a set of n nodes, and Bob gets a perfect matching EB on the same set of nodes. The union EA U EB is a collection of cycles, and the goal is to approximate the number of cycles in this collection. We prove that if Alice is allowed to send o(√n) bits to Bob (and Bob is not allowed to send anything to Alice), then the number of cycles cannot be approximated to within a factor of 1.999, even using a randomized protocol. We prove that it is not even possible to distinguish the case where all cycles are of length 4, from the case where all cycles are of length 8. This lower bound is "natively" one-way: With 4 rounds of communication, it is easy to distinguish these two cases.

https://epubs.siam.org/doi/pdf/10.1137/1.9781611973082.2

The streaming complexity of cycle counting, sorting by reversals, and other problems

In this paper we investigate the problem of building a static data structure that represents a string s using space close to its compressed size, and allows fast access to individual characters of s. This type of data structures was investigated by the recent paper of Bille et al. [3]. Let n be the size of a context-free grammar that derives a unique string s of length L. (Note that L might be exponential in n.) Bille et al. showed a data structure that uses space O(n) and allows to query for the i-th character of s using running time O(logL). Their data structure works on a word RAM with a word size of logL bits.

Data Structure Lower Bounds on Random Access to Grammar-Compressed Strings

Consider a sum-product normed space, ie a space of the form $Y=\ell_1^n \otimes X$, where X is another normed space Each element in Y consists of a length-n vector of elements in X, and the norm of an element in Y is the sum of the norms of its coordinates In this paper we show a constant-distortion embedding from the normed space $\ell_1^n \otimes X$ into a lower-dimensional normed space $\ell_1^{n'} \otimes X$, where n′≪n is some value that depends on the properties of the normed space X (namely, on its Rademacher dimension) In particular, composing this embedding with another well-known embedding of Indyk [18], we get an O(1/e)-distortion embedding from the earth-mover metric EMDΔ on the grid [Δ]2 to $\ell_1^{\Delta^{O(\epsilon)}} \otimes {\sf{EEMD}}_{\Delta^{\epsilon }}$ (where EEMD is a norm that generalizes earth-mover distance) This embedding is stronger (and simpler) than the sketching algorithm of Andoni et al [4], which maps EMDΔ with O(1/e) approximation into sketches of size ΔO(e)

https://www.cse.ust.hk/~qinzhang/papers/icalp12_emd.pdf

Rademacher-sketch : a dimensionality-reducing embedding for sum-product norms, with an application to earth-mover distance

We consider the problem of approximating the minmax value of a multi-player game in strategic form. We argue that in three-player games with 0-1 payoffs, approximating the minmax value within an additive constant smaller than ξ/2, where $\xi = \frac{3-\sqrt5}{2} \approx 0.382$, is not possible by a polynomial time algorithm. This is based on assuming hardness of a version of the so-called planted clique problem in Erdős-Renyi random graphs, namely that of detecting a planted clique. Our results are stated as reductions from a promise graph problem to the problem of approximating the minmax value, and we use the detection problem for planted cliques to argue for its hardness. We present two reductions: a randomised many-one reduction and a deterministic Turing reduction. The latter, which may be seen as a derandomisation of the former, may be used to argue for hardness of approximating the minmax value based on a hardness assumption about deterministic algorithms. Our technique for derandomisation is general enough to also apply to related work about e-Nash equilibria.

Elad Verbin

Papers

The streaming complexity of cycle counting, sorting by reversals, and other problems

Data Structure Lower Bounds on Random Access to Grammar-Compressed Strings

Rademacher-sketch : a dimensionality-reducing embedding for sum-product norms, with an application to earth-mover distance

Approximating the minmax value of three-player games within a constant is as hard as detecting planted cliques