Metagenomics and 16S pyrosequencing have enabled the study of ecosystem structure and dynamics to great depth and accuracy. Co-occurrence and correlation patterns found in these data sets are increasingly used for the prediction of species interactions in environments ranging from the oceans to the human microbiome. In addition, parallelized co-culture assays and combinatorial labelling experiments allow high-throughput discovery of cooperative and competitive relationships between species. In this Review, we describe how these techniques are opening the way towards global ecosystem network prediction and the development of ecosystem-wide dynamic models.

Microbial interactions: from networks to models

We propose an indexing method for time sequences for processing similarity queries. We use the Discrete Fourier Transform (DFT) to map time sequences to the frequency domain, the crucial observation being that, for most sequences of practical interest, only the first few frequencies are strong. Another important observation is Parseval's theorem, which specifies that the Fourier transform preserves the Euclidean distance in the time or frequency domain. Having thus mapped sequences to a lower-dimensionality space by using only the first few Fourier coefficients, we use R * -trees to index the sequences and efficiently answer similarity queries. We provide experimental results which show that our method is superior to search based on sequential scanning. Our experiments show that a few coefficients (1–3) are adequate to provide good performance. The performance gain of our method increases with the number and length of sequences.

/pdf/efficient-similarity-search-in-sequence-databases-1j92qk9o7x.pdf

Efficient Similarity Search In Sequence Databases

In this paper, we address the problem of finding frequent itemsets in a database. Using the closed itemset lattice framework, we show that this problem can be reduced to the problem of finding frequent closed itemsets. Based on this statement, we can construct efficient data mining algorithms by limiting the search space to the closed itemset lattice rather than the subset lattice. Moreover, we show that the set of all frequent closed itemsets suffices to determine a reduced set of association rules, thus addressing another important data mining problem: limiting the number of rules produced without information loss. We propose a new algorithm, called A-Close, using a closure mechanism to find frequent closed itemsets. We realized experiments to compare our approach to the commonly used frequent itemset search approach. Those experiments showed that our approach is very valuable for dense and/or correlated data that represent an important part of existing databases.

/pdf/discovering-frequent-closed-itemsets-for-association-rules-3w1oooufhs.pdf

Discovering Frequent Closed Itemsets for Association Rules

A review on time series data mining

We discuss the video recommendation system in use at YouTube, the world's most popular online video community. The system recommends personalized sets of videos to users based on their activity on the site. We discuss some of the unique challenges that the system faces and how we address them. In addition, we provide details on the experimentation and evaluation framework used to test and tune new algorithms. We also present some of the findings from these experiments.

/pdf/the-youtube-video-recommendation-system-244aw7tv26.pdf

The YouTube video recommendation system

We are given a large database of customer transactions. Each transaction consists of items purchased by a customer in a visit. We present an efficient algorithm that generates all significant assoc...

ImielińskiTomasz

Papers

Mining association rules between sets of items in large databases