This paper introduces a novel algorithm for biological sequence compression that makes use of both statistical properties and repetition within sequences. A panel of experts is maintained to estimate the probability distribution of the next symbol in the sequence to be encoded. Expert probabilities are combined to obtain the final distribution. The resulting information sequence provides insight for further study of the biological sequence. Each symbol is then encoded by arithmetic coding. Experiments show that our algorithm outperforms existing compressors on typical DNA and protein sequence datasets while maintaining a practical running time

/pdf/a-simple-statistical-algorithm-for-biological-sequence-1zmdqh3ufh.pdf

A Simple Statistical Algorithm for Biological Sequence Compression

Motivation: The continuing exponential accumulation of full genome data, including full diploid human genomes, creates new challenges not only for understanding genomic structure, function and evolution, but also for the storage, navigation and privacy of genomic data. Here, we develop data structures and algorithms for the efficient storage of genomic and other sequence data that may also facilitate querying and protecting the data.

Results: The general idea is to encode only the differences between a genome sequence and a reference sequence, using absolute or relative coordinates for the location of the differences. These locations and the corresponding differential variants can be encoded into binary strings using various entropy coding methods, from fixed codes such as Golomb and Elias codes, to variables codes, such as Huffman codes. We demonstrate the approach and various tradeoffs using highly variables human mitochondrial genome sequences as a testbed. With only a partial level of optimization, 3615 genome sequences occupying 56 MB in GenBank are compressed down to only 167 KB, achieving a 345-fold compression rate, using the revised Cambridge Reference Sequence as the reference sequence. Using the consensus sequence as the reference sequence, the data can be stored using only 133 KB, corresponding to a 433-fold level of compression, roughly a 23% improvement. Extensions to nuclear genomes and high-throughput sequencing data are discussed.

Availability: Data are publicly available from GenBank, the HapMap web site, and the MITOMAP database. Supplementary materials with additional results, statistics, and software implementations are available from http://mammag.web.uci.edu/bin/view/Mitowiki/ProjectDNACompression.

Contact: pfbaldi@ics.uci.edu

Data structures and compression algorithms for genomic sequence data

Research in the genomic sciences is confronted with the volume of sequencing and resequencing data increasing at a higher pace than that of data storage and communication resources, shifting a significant part of research budgets from the sequencing component of a project to the computational one. Hence, being able to efficiently store sequencing and resequencing data is a problem of paramount importance. In this article, we describe GReEn (Genome Resequencing Encoding), a tool for compressing genome resequencing data using a reference genome sequence. It overcomes some drawbacks of the recently proposed tool GRS, namely, the possibility of compressing sequences that cannot be handled by GRS, faster running times and compression gains of over 100-fold for some sequences. This tool is freely available for non-commercial use at ftp://ftp.ieeta.pt/~ap/codecs/GReEn1.tar.gz.

/pdf/green-a-tool-for-efficient-compression-of-genome-2qhw5ee2mb.pdf

GReEn: a tool for efficient compression of genome resequencing data

The main strategy of algorithms in this class is to substitute “repeated strings”, i.e., exact replicas of a substring in a string, within the text with a convenient coding scheme. Those methods generally fall into the Macro Model for Data Compression introduced and studied by Storer and Szymanski (1982). The well known algorithms by Ziv and Lempel (1977, 1978), as well as many others, turn out to be instances of that paradigm, which is based on the principle of finding exact repeated substrings in a string and replacing them by pointers to a common copy. That is, redundancy in a string is translated into the notion of exact replicas of substrings. For instance, the Ziv and Lempel (1977) algorithm first parses the input string into distinct phrases and then each phrase is encoded by a triplet of numbers. An example of the parsing rule is provided in Fig. 1.

http://www.math.unipa.it/~raffaele/suppMaterial/compReview/CompressBioInfo/Index_files/Supplementary_Comp_bioinformatics.pdf

Supplementary Material for Textual Data Compression in Computational Biology: A synopsis

Motivation: Textual data compression, and the associated techniques coming from information theory, are often perceived as being of interest for data communication and storage. However, they are also deeply related to classification and data mining and analysis. In recent years, a substantial effort has been made for the application of textual data compression techniques to various computational biology tasks, ranging from storage and indexing of large datasets to comparison and reverse engineering of biological networks. Results: The main focus of this review is on a systematic presentation of the key areas of bioinformatics and computational biology where compression has been used. When possible, a unifying organization of the main ideas and techniques is also provided. Availability: It goes without saying that most of the research results reviewed here offer software prototypes to the bioinformatics community. The supplementary material (see next) provides pointers to software and benchmark datasets for a range of applications of broad interest. Contact: raffaele@math.unipa.it Supplementary: In addition to provide reference to software, the supplementary material also gives a brief presentation of some fundamental results and techniques related to this paper. It is at: http://www.math.unipa.it/˜raffaele/suppMaterial/ compReview/

/pdf/textual-data-compression-in-computational-biology-a-synopsis-2x2r724zwl.pdf

Textual Data Compression In Computational Biology: A synopsis

Standard compression algorithms are not able to compress DNA sequences. Recently, new algorithms have been introduced specifically for this purpose, often using detection of long approximate repeats. In this paper, we present another algorithm, DNAPack, based on dynamic programming. In comparison with former existing programs, it compresses DNA slightly better, while the cost of dynamic programming is almost negligible.

Fabrice Le Fessant

Papers

DNA compression challenge revisited: a dynamic programming approach