Proceedings ArticleDOI
On the resemblance and containment of documents
TLDR
The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that could be done independently for each document.Abstract:
Given two documents A and B we define two mathematical notions: their resemblance r(A, B) and their containment c(A, B) that seem to capture well the informal notions of "roughly the same" and "roughly contained." The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that can be done independently for each document. Furthermore, the resemblance can be evaluated using a fixed size sample for each document. This paper discusses the mathematical properties of these measures and the efficient implementation of the sampling process using Rabin (1981) fingerprints.read more
Citations
More filters
Journal ArticleDOI
Coordinated weighted sampling for estimating aggregates over multiple weight assignments
TL;DR: A sampling framework based on coordinated weighted samples that is suited for multiple weight assignments is developed and estimators that are orders of magnitude tighter than previously possible are obtained.
Proceedings ArticleDOI
Dynamic K-Gram Based Software Birthmark
TL;DR: A novel dynamic birthmark is proposed that is more resilient to semantics-preserving transformations than the static k-gram birthmark and is compared with static approach from similarity with academic obfuscation tools.
Journal ArticleDOI
WAN-optimized replication of backup datasets using stream-informed delta compression
TL;DR: In this paper, the authors present a new technique for replicating backup datasets across a wide area network (WAN) that not only eliminates duplicate regions of files (deduplication) but also compresses similar areas of files with delta compression, which is available as a feature of EMC Data Domain systems.
Proceedings ArticleDOI
Orthogonal query recommendation
TL;DR: This work intentionally seeks out orthogonal queries, which are related queries that have (almost) no common terms with the user's query, which allows an Orthogonal query to satisfy the users' informational need when small perturbations of the original keyword set are insufficient.
On the complexity of unique games and graph expansion
Sanjeev Arora,David Steurer +1 more
TL;DR: It is shown that the Unique Games Conjecture is true if the expansion of small sets in graphs is hard to approximate in a certain regime, the first sufficient condition for the truth of the conjecture based on the inapproximability of a natural combinatorial problem.
References
More filters
Book
The Probabilistic Method
TL;DR: A particular set of problems - all dealing with “good” colorings of an underlying set of points relative to a given family of sets - is explored.
Journal ArticleDOI
Syntactic clustering of the Web
TL;DR: An efficient way to determine the syntactic similarity of files is developed and applied to every document on the World Wide Web, and a clustering of all the documents that are syntactically similar is built.
Journal ArticleDOI
Min-Wise Independent Permutations
TL;DR: This research was motivated by the fact that such a family of permutations is essential to the algorithm used in practice by the AltaVista web index software to detect and filter near-duplicate documents.
Proceedings Article
Finding similar files in a large file system
TL;DR: Application of sif can be found in file management, information collecting, program reuse, file synchronization, data compression, and maybe even plagiarism detection.
Proceedings ArticleDOI
Copy detection mechanisms for digital documents
TL;DR: This paper proposes a system for registering documents and then detecting copies, either complete copies or partial copies, and describes algorithms for such detection, and metrics required for evaluating detection mechanisms (covering accuracy, efficiency, and security).