Proceedings ArticleDOI
On the resemblance and containment of documents
TLDR
The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that could be done independently for each document.Abstract:
Given two documents A and B we define two mathematical notions: their resemblance r(A, B) and their containment c(A, B) that seem to capture well the informal notions of "roughly the same" and "roughly contained." The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that can be done independently for each document. Furthermore, the resemblance can be evaluated using a fixed size sample for each document. This paper discusses the mathematical properties of these measures and the efficient implementation of the sampling process using Rabin (1981) fingerprints.read more
Citations
More filters
Journal ArticleDOI
A Survey and Classification of Storage Deduplication Systems
João Paulo,José Pereira +1 more
TL;DR: A classification of deduplication systems according to six criteria that correspond to key design decisions: granularity, locality, timing, indexing, technique, and scope is identified and describes the different approaches used for each.
Journal ArticleDOI
Table union search on open data
TL;DR: This work defines the table union search problem and presents a probabilistic solution for finding tables that are unionable with a query table within massive repositories, and proposes a data-driven approach that automatically determines the best model to use for each pair of attributes.
Patent
Systems and methods for the utilization of metadata for synchronization optimization
TL;DR: In this paper, the authors present an approach for intra-community-only change tracking optimization, where each peer only tracks changes for itself and other peers within its change community (or communities) (a.k.a., "Intra-Community-Only Change Tracking Optimization").
Proceedings ArticleDOI
Detectives: detecting coalition hit inflation attacks in advertising networks streams
TL;DR: This paper describes the advertising network model, and focuses on the most sophisticated type of fraud, which involves coalitions among fraudsters, and builds on several published theoretical results to devise the Similarity-Seeker algorithm that discovers coalitions made by pairs of fraudsters.
Proceedings ArticleDOI
What's the difference?: efficient set reconciliation without prior context
TL;DR: A synopsis structure that allows two nodes to compute the elements belonging to the set difference in a single round with communication overhead proportional to the size of the difference times the logarithm of the keyspace is described.
References
More filters
Book
The Probabilistic Method
TL;DR: A particular set of problems - all dealing with “good” colorings of an underlying set of points relative to a given family of sets - is explored.
Journal ArticleDOI
Syntactic clustering of the Web
TL;DR: An efficient way to determine the syntactic similarity of files is developed and applied to every document on the World Wide Web, and a clustering of all the documents that are syntactically similar is built.
Journal ArticleDOI
Min-Wise Independent Permutations
TL;DR: This research was motivated by the fact that such a family of permutations is essential to the algorithm used in practice by the AltaVista web index software to detect and filter near-duplicate documents.
Proceedings Article
Finding similar files in a large file system
TL;DR: Application of sif can be found in file management, information collecting, program reuse, file synchronization, data compression, and maybe even plagiarism detection.
Proceedings ArticleDOI
Copy detection mechanisms for digital documents
TL;DR: This paper proposes a system for registering documents and then detecting copies, either complete copies or partial copies, and describes algorithms for such detection, and metrics required for evaluating detection mechanisms (covering accuracy, efficiency, and security).