Proceedings ArticleDOI
On the resemblance and containment of documents
TLDR
The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that could be done independently for each document.Abstract:
Given two documents A and B we define two mathematical notions: their resemblance r(A, B) and their containment c(A, B) that seem to capture well the informal notions of "roughly the same" and "roughly contained." The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that can be done independently for each document. Furthermore, the resemblance can be evaluated using a fixed size sample for each document. This paper discusses the mathematical properties of these measures and the efficient implementation of the sampling process using Rabin (1981) fingerprints.read more
Citations
More filters
Proceedings ArticleDOI
Summarizing data using bottom-k sketches
Edith Cohen,Haim Kaplan +1 more
TL;DR: It is shown that k-mins sketches can be derived from respective bottom-K sketches, which enables the use of bottom-k sketches with off-the-shelf k-min estimators, and develops and analyze data structures that incrementally construct bottom-c sketches and all-distances bottom- k sketches.
Proceedings ArticleDOI
Enabling search over encrypted multimedia databases
TL;DR: Wang et al. as discussed by the authors proposed a secure indexing scheme for content-based retrieval over encrypted multimedia databases, which is first encrypted by the content owner and then stored onto the server.
Journal ArticleDOI
Improving duplicate elimination in storage systems
TL;DR: A new object partitioning technique is proposed, called fingerdiff, that improves upon existing schemes in several important respects, most notably, fingerdiff dynamically chooses a partitioning strategy for a data object based on its similarities with previously stored objects in order to improve storage and bandwidth utilization.
Proceedings ArticleDOI
Three-level caching for efficient query processing in large Web search engines
Xiaohui Long,Torsten Suel +1 more
TL;DR: This work proposes and evaluates a three-level caching scheme that adds an intermediate level of caching for additional performance gains, and proposes and study several offline and online algorithms for the resulting weighted caching problem, which turns out to be surprisingly rich in structure.
Journal ArticleDOI
Informed content delivery across adaptive overlay networks
TL;DR: This work makes the case for an erasure-resilient encoding of the content, and demonstrates the performance benefits of informed content-delivery mechanisms and how they complement existing overlay network architectures.
References
More filters
Book
The Probabilistic Method
TL;DR: A particular set of problems - all dealing with “good” colorings of an underlying set of points relative to a given family of sets - is explored.
Journal ArticleDOI
Syntactic clustering of the Web
TL;DR: An efficient way to determine the syntactic similarity of files is developed and applied to every document on the World Wide Web, and a clustering of all the documents that are syntactically similar is built.
Journal ArticleDOI
Min-Wise Independent Permutations
TL;DR: This research was motivated by the fact that such a family of permutations is essential to the algorithm used in practice by the AltaVista web index software to detect and filter near-duplicate documents.
Proceedings Article
Finding similar files in a large file system
TL;DR: Application of sif can be found in file management, information collecting, program reuse, file synchronization, data compression, and maybe even plagiarism detection.
Proceedings ArticleDOI
Copy detection mechanisms for digital documents
TL;DR: This paper proposes a system for registering documents and then detecting copies, either complete copies or partial copies, and describes algorithms for such detection, and metrics required for evaluating detection mechanisms (covering accuracy, efficiency, and security).