We introduce an alternative Lempel-Ziv text parsing, LZ-End, that converges to the entropy and in practice gets very close to LZ77. LZ-End forces sources to finish at the end of a previous phrase. Most Lempel-Ziv parsings can decompress the text only from the beginning. LZ-End is the only parsing we know of able of decompressing arbitrary phrases in optimal time, while staying closely competitive with LZ77, especially on highly repetitive collections, where LZ77 excells. Thus LZ-End is ideal as a compression format for highly repetitive sequence databases, where access to individual sequences is required, and it also opens the door to compressed indexing schemes for such collections.

/pdf/lz77-like-compression-with-fast-random-access-254xk9xtv0.pdf

LZ77-Like Compression with Fast Random Access

A run (or a maximal repetition) in a string is an inclusion-maximal periodic segment in a string. Let p(n) be the maximal number of runs in a string of length n. It has been shown in [8] that p(n) = O(n), the proof was very complicated and the constant coefficient in O(n) has not been given explicitly. We propose a new approach to the analysis of runs based on the properties of subperiods: the periods of periodic parts of the runs. We show that p(n) < 5 n. Our proof is inspired by the results of [4], where the role of new periodicity lemmas has been emphasized.

The number of runs in a string : Improved analysis of the linear upper bound

We give a space-efficient simple algorithm for computing the Lempel-Ziv factorization of a string. For a string of length n over an integer alphabet, it runs in O(n) time independently of alphabet size and uses o(n) additional space.

/pdf/a-simple-algorithm-for-computing-the-lempel-ziv-2lcsztcwh2.pdf

A Simple Algorithm for Computing the Lempel Ziv Factorization

https://hal-upec-upem.archives-ouvertes.fr/hal-00619712/document

Maximal repetitions in strings

The article is an overview of basic issues related to repetitions in strings, concentrating on algorithmic and combinatorial aspects. This area is important both from theoretical and practical points of view. Repetitions are highly periodic factors (substrings) in strings and are related to periodicities, regularities, and compression. The repetitive structure of strings leads to higher compression rates, and conversely, some compression techniques are at the core of fast algorithms for detecting repetitions. There are several types of repetitions in strings: squares, cubes, and maximal repetitions also called runs. For these repetitions, we distinguish between the factors (sometimes qualified as distinct) and their occurrences (also called positioned factors). The combinatorics of repetitions is a very intricate area, full of open problems. For example we know that the number of (distinct) primitively-rooted squares in a string of length n is no more than [email protected](logn), conjecture to be n, and that their number of occurrences can be @Q(nlogn). Similarly we know that there are at most 1.029n and at least 0.944n maximal repetitions and the conjecture is again that the exact bound is n. We know almost everything about the repetitions in Sturmian words, but despite the simplicity of these words, the results are nontrivial. One of the main motivations for writing this text is the development during the last couple of years of new techniques and results about repetitions. We report both the progress which has been achieved and which we expect to happen.

/pdf/repetitions-in-strings-algorithms-and-combinatorics-21vlb37570.pdf

Repetitions in strings: Algorithms and combinatorics

A border of a string x is a proper (but possibly empty) prefix of x that is also a suffix of x. The border array β = β[1..n] of a string x = x[1..n] is an array of nonnegative integers in which each element β[i], 1 ≤ i ≤ n, is the length of the longest border of x[1..i]. In this paper we first present a simple linear-time algorithm to determine whether or not a given array y = y[1..n] of integers is a border array of some string on an alphabet of unbounded size. We state as an open problem the design of a corresponding and equally efficient algorithm on an alphabet of bounded size α. We then consider the problem of generating all possible distinct border arrays of given length n on a bounded or unbounded alphabet, and doing so in time proportional to the number of arrays generated. A previously published algorithm that claims to solve this problem in constant time per array generated is shown to be incorrect, and new algorithms are proposed. We state as open the design of an equally efficient on-line algorithm for this problem.

/pdf/verifying-a-border-array-in-linear-time-pktp4a0mhx.pdf

Verifying a border array in linear time

nor x[i−|u|..i−1] equals u. We say the repetition has period |u| and exponent e; it can be specified by the integer triple (i, |u|, e). It is well known [2] that the maximum number of repetitions in a string x = x[1..n] is Θ(n log n), and that the number of repetitions in x can be computed in Θ(n log n) time [2, 1, 10]. A string u is a run iff it is periodic of (minimum) period p ≤ |u|/2. Thus x = abaabaabaabaab = (aba)4ab is a run of period |aba| = 3. A substring u = x[i..j] of x is a run or maximal periodicity in x iff it is a run of period p and neither x[i−1..j] nor x[i..j+1] is a run of period p. The run u has exponent e = b|u|/pc and possibly empty tail t = x[i+ep..j] (proper prefix of x[i..i+p−1]). Thus

The maximum number of runs in a string

We describe an algorithm that identifies all the repeating substrings (tandem, overlapping, and split) in a given string x = X[1..n]. Given the suffix arrays of x and of the reversed string x, the algorithm requires Θ(n) time for its execution and represents its output in Θ(n) space, either as a reduced suffix array (called an NE array) or as a reduced suffix tree (called an NE tree). The output substrings u are nonextendible (NE); that is, any extension of some occurrence of u in x, either to the left or to the right, yields a string (λu or uλ) that is unequal to the same extension of some other occurrence of u. Thus the number of substrings output is the minimum required to identify all the repeating substrings in x. The output can be used in a straightforward way to identify only repeating substrings that satisfy some proximity or minimum length condition.

Computing all repeats using suffix arrays

We introduce quasi suffix arrays as a generalization of suffix arrays for character strings. We show that a quasi suffix array encodes enough of the structure of the string to be a useful construct for many applications where the full power of suffix arrays is not necessary, notably in problems that do not require lexicographical order, for example, pattern-matching or calculation of repeating substrings. We are interested in quasi suffix arrays, for we believe that they can be calculated by simple, fast, and space efficient algorithms. As a first step towards this goal, we describe a family DIST of algorithms (inspired by the Crochemore's repetitions algorithm) that compute the quasi suffix array in the average-case in O(|x| log |x|) time, where x is the input string. Based on experiments conducted by one of us (Xiao), it appears that in practice our algorithms execute faster than all suffix tree and most suffix array construction algorithms. Though at this time we can only prove that the average-case complexity is O(|x| log |x|), tests carried out by one of us (Holub) strongly suggest that not only the worst-case complexity may be the same as the average-case complexity, but both may in fact be linear. Given the very recent results on computing suffix arrays in linear time by recursive algorithms, the only advantage quasi suffix arrays can have lies in the simplicity and space efficiency of DIST algorithms that do not use recursion.

F. Franěk

Papers

Verifying a border array in linear time

The maximum number of runs in a string

Computing all repeats using suffix arrays

Computing quasi suffix arrays

Repetitions in Sturmian strings