Proceedings ArticleDOI
On the resemblance and containment of documents
TLDR
The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that could be done independently for each document.Abstract:
Given two documents A and B we define two mathematical notions: their resemblance r(A, B) and their containment c(A, B) that seem to capture well the informal notions of "roughly the same" and "roughly contained." The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that can be done independently for each document. Furthermore, the resemblance can be evaluated using a fixed size sample for each document. This paper discusses the mathematical properties of these measures and the efficient implementation of the sampling process using Rabin (1981) fingerprints.read more
Citations
More filters
Patent
Optimizing signature computation and sampling for fast adaptive similarity detection based on algorithm-specific performance
Qing Yang,Pulkit Misra +1 more
TL;DR: A set of similarity detection algorithms and techniques for determining which signature calculation, sampling, and generation algorithms may be most beneficially applied to application related data are described in this paper, which work well with SSD caching software to product high speed, high accuracy, and low false positive detections.
Posted ContentDOI
Fast genome and metagenome distance estimation using MinHash
Brian D. Ondov,Todd J. Treangen,Adam B. Mallonee,Nicholas H. Bergman,Sergey Koren,Adam M. Phillippy +5 more
TL;DR: Mash extends the MinHash dimensionality-reduction technique to include a pairwise mutation distance and P-value significance test, enabling the efficient clustering and search of massive sequence collections.
Book ChapterDOI
Hashing-Based Distributed Multi-party Blocking for Privacy-Preserving Record Linkage
TL;DR: This work proposes a novel blocking approach for multi-party PPRL to efficiently and effectively prune the record sets that are unlikely to match and provides an analysis of the technique in terms of complexity, quality, and privacy.
Proceedings ArticleDOI
Finding "Who Is Talking to Whom" in VoIP Networks via Progressive Stream Clustering
Olivier Verscheure,Michail Vlachos,Aris Anagnostopoulos,Pascal Frossard,Eric Bouillet,Philip S. Yu +5 more
TL;DR: This work studies the feasibility of revealing pairs of anonymous and encrypted conversing parties (caller/callee pair of streams) by exploiting the vulnerabilities inherent to VoIP systems by trivializing each VoIP stream into a binary time-series.
Posted Content
Preserving Semantic Neighborhoods for Robust Cross-modal Retrieval.
TL;DR: This work proposes novel within-modality losses which encourage semantic coherency in both the text and image subspaces, which does not necessarily align with visual cohe consistency.
References
More filters
Book
The Probabilistic Method
TL;DR: A particular set of problems - all dealing with “good” colorings of an underlying set of points relative to a given family of sets - is explored.
Journal ArticleDOI
Syntactic clustering of the Web
TL;DR: An efficient way to determine the syntactic similarity of files is developed and applied to every document on the World Wide Web, and a clustering of all the documents that are syntactically similar is built.
Journal ArticleDOI
Min-Wise Independent Permutations
TL;DR: This research was motivated by the fact that such a family of permutations is essential to the algorithm used in practice by the AltaVista web index software to detect and filter near-duplicate documents.
Proceedings Article
Finding similar files in a large file system
TL;DR: Application of sif can be found in file management, information collecting, program reuse, file synchronization, data compression, and maybe even plagiarism detection.
Proceedings ArticleDOI
Copy detection mechanisms for digital documents
TL;DR: This paper proposes a system for registering documents and then detecting copies, either complete copies or partial copies, and describes algorithms for such detection, and metrics required for evaluating detection mechanisms (covering accuracy, efficiency, and security).