Motivation: Chimeric DNA sequences often form during polymerase chain reaction amplification, especially when sequencing single regions (e.g. 16S rRNA or fungal Internal Transcribed Spacer) to assess diversity or compare populations. Undetected chimeras may be misinterpreted as novel species, causing inflated estimates of diversity and spurious inferences of differences between populations. Detection and removal of chimeras is therefore of critical importance in such experiments.

Results: We describe UCHIME, a new program that detects chimeric sequences with two or more segments. UCHIME either uses a database of chimera-free sequences or detects chimeras de novo by exploiting abundance data. UCHIME has better sensitivity than ChimeraSlayer (previously the most sensitive database method), especially with short, noisy sequences. In testing on artificial bacterial communities with known composition, UCHIME de novo sensitivity is shown to be comparable to Perseus. UCHIME is >100× faster than Perseus and >1000× faster than ChimeraSlayer.

Contact: [email protected]

Availability: Source, binaries and data: http://drive5.com/uchime.

Supplementary information:Supplementary data are available at Bioinformatics online.

/pdf/uchime-improves-sensitivity-and-speed-of-chimera-detection-3ear51c4ml.pdf

UCHIME improves sensitivity and speed of chimera detection

The lion's share of bacteria in various environments cannot be cloned in the laboratory and thus cannot be sequenced using existing technologies. A major goal of single-cell genomics is to complement gene-centric metagenomic data with whole-genome assemblies of uncultivated organisms. Assembly of single-cell data is challenging because of highly non-uniform read coverage as well as elevated levels of sequencing errors and chimeric reads. We describe SPAdes, a new assembler for both single-cell and standard (multicell) assembly, and demonstrate that it improves on the recently released E+V-SC assembler (specialized for single-cell data) and on popular assemblers Velvet and SoapDeNovo (for multicell data). SPAdes generates single-cell assemblies, providing information about genomes of uncultivatable bacteria that vastly exceeds what may be obtained via traditional metagenomics studies. SPAdes is available online ( http://bioinf.spbau.ru/spades ). It is distributed as open source software.

SPAdes, a new genome assembly algorithm and its applications to single-cell sequencing ( 7th Annual SFAF Meeting, 2012)

Digital watermarking is a key ingredient to copyright protection. It provides a solution to illegal copying of digital material and has many other useful applications such as broadcast monitoring and the recording of electronic transactions. Now, for the first time, there is a book that focuses exclusively on this exciting technology. Digital Watermarking covers the crucial research findings in the field: it explains the principles underlying digital watermarking technologies, describes the requirements that have given rise to them, and discusses the diverse ends to which these technologies are being applied. As a result, additional groundwork is laid for future developments in this field, helping the reader understand and anticipate new approaches and applications.

Digital Watermarking

The Ribosomal Database Project (RDP) is a curated database that offers ribosome-related data, analysis services and associated computer programs. The offerings include phylogenetically ordered alignments of ribosomal RNA (rRNA) sequences, derived phylogenetic trees, rRNA secondary structure diagrams, and various software for handling, analyzing and displaying alignments and trees. The data are available via anonymous FTP (rdp.life.uiuc.edu), electronic mail (server@rdp.life.uiuc.edu), gopher (rdpgopher.life.uiuc.edu) and WWW (http://rdpwww.life.uiuc.edu/ ). The electronic mail and WWW servers provide ribosomal probe checking, approximate phylogenetic placement of user-submitted sequences, screening for possible chimeric rRNA sequences, automated alignment, and a suggested placement of an unknown sequence on an existing phylogenetic tree.

/pdf/the-rdp-ribosomal-database-project-4uwm5exz1v.pdf

The rdp (ribosomal database project)

We explore the effect of dimensionality on the nearest neighbor problem. We show that under a broad set of conditions (much broader than independent and identically distributed dimensions), as dimensionality increases, the distance to the nearest data point approaches the distance to the farthest data point. To provide a practical perspective, we present empirical results on both real and synthetic data sets that demonstrate that this effect can occur for as few as 10-15 dimensions. These results should not be interpreted to mean that high-dimensional indexing is never meaningful; we illustrate this point by identifying some high-dimensional workloads for which this effect does not occur. However, our results do emphasize that the methodology used almost universally in the database literature to evaluate high-dimensional indexing techniques is flawed, and should be modified. In particular, most such techniques proposed in the literature are not evaluated versus simple linear scan, and are evaluated over workloads for which nearest neighbor is not meaningful. Often, even the reported experiments, when analyzed carefully, show that linear scan would outperform the techniques being proposed on the workloads studied in high (10-15) dimensionality!.

When is nearest neighbor meaningful

In the domain of biological classification, classifications are performed hierarchically. There are no standard classifications which are unanimously accepted by the community of each domain; many different interacting views of classification exist about the same data, and the discovery of new data results in changes to the existing classification. Even a single individual may change his or her own classification of a particular group. Since multiple classification views interact, they are semantically related. It is difficult to model this kind of dynamically evolving and semantically interacting classification system using traditional data models, which lack the structural flexibility necessary to support dynamic views of hierarchic classifications, and cannot properly capture the history of these complex interactions. We have developed a new data model which is suitable for supporting semantically interacting dynamic views of hierarchic biological classifications. On the basis of our new data model we have developed a prototype database system called HICLAS (HIerarchical CLAssification System); its domain is plant taxonomy. HICLAS is available through the Internet and an X-window interface has been implemented to support queries to classification data.

A new data model for biological classification

The vast increase in DNA sequencing capacity over the last decade has quickly turned biology into a dataintensive science. Nevertheless, current sequencers such as Illumia HiSeq have high random per-base error rates, which makes sequencing error correction an indispensable requirement for many sequence analysis applications. Most existing error correction methods demand large expensive memory space, which limits their scalability for handling large datasets. In this paper, we present a new disk based method, called DiskBQcor, for sequencing error correction. DiskBQcor stores k-mers of sequencing genome data along with their associated metadata on inexpensive disk and utilizes a disk based index tree to efficiently process special box queries to obtain relevant k-mers and their occurring frequencies. It then applies a comprehensive voting mechanism and possibly an efficient binary encoding based assembly technique to verify and correct an erroneous base in a genome sequence under various conditions. Our experiments demonstrate that the proposed method is quite promising in error verification and correction for sequencing genome data on disk. keywords: DNA sequencing, error correction, index tree, box query, algorithm.

Using disk based index and box queries for genome sequencing error correction

Clustering in continuous vector data spaces is a well-studied problem. In recent years there has been a significant amount of research work in clustering categorical data. However, most of these works deal with market-basket type transaction data and are not specifically optimized for high-dimensional vectors. Our focus in this paper is to efficiently cluster high-dimensional vectors in non-ordered discrete data spaces (NDDS). We have defined several necessary geometrical concepts in NDDS which form the basis of our clustering algorithm. Several new heuristics have been employed exploiting the characteristics of vectors in NDDS. Experimental results on large synthetic datasets demonstrate that the proposed approach is effective, in terms of cluster quality, robustness and running time. We have also applied our clustering algorithm to real datasets with promising results.

/pdf/clustering-non-ordered-discrete-data-6fto65t9sz.pdf

Clustering Non-Ordered Discrete Data *

Applications demanding multidimensional index structures for performing efficient similarity queries often involve a large amount of data The conventional tuple-loading approach to building such an index structure for a large data set is inefficient To overcome the problem, a number of algorithms to bulk-load the index structures, like the R-tree, from scratch for large data sets in continuous data spaces have been proposed However, many of them cannot be directly applied to a non-ordered discrete data space (NDDS) where data values on each dimension are discrete and have no natural ordering No bulk-loading algorithm has been developed specifically for an index structure, such as the ND-tree, in an NDDS In this paper, we present a bulk-loading algorithm, called the NDTBL, for the ND-tree in NDDSs It adopts a special in-memory structure to efficiently construct the target ND-tree It utilizes and extends some operations in the original ND-tree tuple-loading algorithm to exploit the properties of an NDDS in choosing and splitting data sets/nodes during the bulk-loading process It also employs some strategies such as multi-way splitting and memory buffering to enhance efficiency Our experimental studies show that the presented algorithm is quite promising in bulk-loading the ND-tree for large data sets in NDDSs

/pdf/bulk-loading-the-nd-tree-in-non-ordered-discrete-data-spaces-40u5tn8m3p.pdf

Bulk-loading the ND-tree in non-ordered discrete data spaces

Nearest Neighbor search is a fundamental task in many applications. At present state of the art, approaches to nearest neighbor search are not efficient in high dimensions. In this paper we present an efficient angle based balanced index structure called AB-tree, which uses heuristics to decide whether or not to access a node in the index tree based on the estimated angle and the weight of the node. We have presented the result of AB-tree for up to 64 dimensions, and have shown that the performance of the AB-tree algorithm does not deteriorate when processing nearest neighbor queries as dimension increases. However, it is no longer guaranteed that the entire true K nearest neighbors to a query point will be found. Extensive experiments on synthetic data and real data demonstrate that the search time is improved by a factor of up to 85 times over that of SS-tree in 64 dimension while maintaining 90 percent accuracy. Real data includes color histogram and corner like features in a heterogenous collection of natural images.

Sakti Pramanik

Papers

A new data model for biological classification

Using disk based index and box queries for genome sequencing error correction

Clustering Non-Ordered Discrete Data *

Bulk-loading the ND-tree in non-ordered discrete data spaces

Efficient search scheme for very large image databases