# Adapting the Knuth-Morris-Pratt algorithm for pattern matching in Huffman encoded texts

TL;DR: A bitwise KMP algorithm is proposed that can move one extra bit in the case of a mismatch since the alphabet is binary, and two practical Huffman decoding schemes which handle more than a single bit per machine operation are combined.

Abstract: In the present work we perform compressed pattern matching in binary Huffman encoded texts [Huffman, D. (1952). A method for the construction of minimum redundancy codes, Proc. of the IRE, 40, 1098-1101]. A modified Knuth-Morris-Pratt algorithm is used in order to overcome the problem of false matches, i.e., an occurrence of the encoded pattern in the encoded text that does not correspond to an occurrence of the pattern itself in the original text. We propose a bitwise KMP algorithm that can move one extra bit in the case of a mismatch since the alphabet is binary. To avoid processing any bit of the encoded text more than once, a preprocessed table is used to determine how far to back up when a mismatch is detected, and is defined so that we are always able to align the start of the encoded pattern with the start of a codeword in the encoded text. We combine our KMP algorithm with two practical Huffman decoding schemes which handle more than a single bit per machine operation; skeleton trees defined by Klein [Klein, S. T. (2000). Skeleton trees for efficient decoding of huffman encoded texts. Information Retrieval, 3, 7-23], and numerical comparisons between special canonical values and portions of a sliding window presented in Moffat and Turpin [Moffat, A., & Turpin, A. (1997). On the implementation of minimum redundancy prefix codes. IEEE Transactions on Communications, 45, 1200-1207]. Experiments show rapid search times of our algorithms compared to the "decompress then search" method, therefore, files can be kept in their compressed form, saving memory space. When compression gain is important, these algorithms are better than cgrep [Ferragina, P., Tommasi, A., & Manzini, G. (2004). C Library to search over compressed texts, http://roquefort.di.unipi.it/~ferrax/CompressedSearch], which is only slightly faster than ours.

## Summary (1 min read)

### Summary

- A modified Knuth-Morris-Pratt (KMP) algorithm is used in order to overcome the problem of false matches, i.e., an occurrence of the encoded pattern in the encoded text that does not correspond to an occurrence of the pattern itself in the original text.
- The authors propose a bitwise KMP algorithm that can move one extra bit in the case of a mismatch, since the alphabet is binary.
- To avoid processing any encoded text bit more than once, a preprocessed table is used to determine how far to back up when a mismatch is detected, and is defined so that the encoded pattern is always aligned with the start of a codeword in the encoded text.
- The authors call the combined algorithms sk-kmp and win-kmp respectively.
- The following table compares their algorithms with cgrep of Moura et al. [2] and agrep which searches the uncompressed text.
- Columns three and four compare the compression performance (size of the compressed text as a percentage of the uncompressed text) of the Huffman code (huff ) with cgrep.
- The next columns compare the processing time of pattern matching of these algorithms.
- The “decompress and search” methods, which decode using skeleton trees or Moffat and Turpin’s sliding window and search in parallel using agrep, are called sk-d and win-d respectively.
- The search times are average values for patterns ranging from infrequent to frequent ones.

Did you find this useful? Give us your feedback

...read more

##### Citations

26 citations

### Additional excerpts

...An advantage of this technique is that it allows for higher compression ratio than the lossless [3,4]....

[...]

24 citations

21 citations

### Cites methods from "Adapting the Knuth-Morris-Pratt alg..."

...This has also been applied on Huffman trees [20] producing a compact tree for efficient use, such as compressed pattern matching [29]....

[...]

14 citations

14 citations

### Cites methods from "Adapting the Knuth-Morris-Pratt alg..."

...Skeleton trees have been used to accelerate compressed pattern matching in [23]....

[...]

##### References

5,013 citations

3,023 citations

782 citations

271 citations

### "Adapting the Knuth-Morris-Pratt alg..." refers background in this paper

...[2] and agrep which searches the uncompressed text....

[...]

222 citations