Proceedings ArticleDOI
On the resemblance and containment of documents
TLDR
The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that could be done independently for each document.Abstract:
Given two documents A and B we define two mathematical notions: their resemblance r(A, B) and their containment c(A, B) that seem to capture well the informal notions of "roughly the same" and "roughly contained." The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that can be done independently for each document. Furthermore, the resemblance can be evaluated using a fixed size sample for each document. This paper discusses the mathematical properties of these measures and the efficient implementation of the sampling process using Rabin (1981) fingerprints.read more
Citations
More filters
Patent
Systems, devices, and/or methods for managing data
Edith Cohen,Haim Kaplan +1 more
TL;DR: In this article, the authors propose a method to automatically store and compute a sketch of a dataset that supports an automatically determined estimator of properties of the dataset, which can be related to any population.
Posted ContentDOI
Raven: a de novo genome assembler for long reads
TL;DR: New methods for the improvement of long-read de novo genome assembly incorporated into a straightforward tool called Raven, which is one of two fastest, it reconstructs the sequenced genome in the least amount of fragments, has better or comparable accuracy, and maintains similar performance for various genomes.
Posted Content
Approximate Nearest Neighbor Search in High Dimensions
TL;DR: The nearest neighbor problem as mentioned in this paper is defined as follows: given a set of points in some metric space, build a data structure that, given any point $q, returns a point in the set $P$ that is closest to $q$ (its "nearest neighbor" in $P), which is then used to find the nearest neighbor without computing all distances between the points.
Proceedings ArticleDOI
SLS at SemEval-2016 Task 3: Neural-based Approaches for Ranking in Community Question Answering.
Mitra Mohtarami,Yonatan Belinkov,Wei-Ning Hsu,Yu Zhang,Tao Lei,Kfir Bar,Scott Cyphers,James Glass +7 more
TL;DR: A bag-of-vectors approach with various vectorand text-based features, and different neural network approaches including CNNs and LSTMs to capture the semantic similarity between questions and answers for ranking purpose is developed.
Proceedings ArticleDOI
Malware Classification and Class Imbalance via Stochastic Hashed LZJD
Edward Raff,Charles Nicholas +1 more
TL;DR: This work develops the new SHWeL feature vector representation, by extending the recently proposed Lempel-Ziv Jaccard Distance, which provides significantly improved accuracy while reducing algorithmic complexity to O(N).
References
More filters
Book
The Probabilistic Method
TL;DR: A particular set of problems - all dealing with “good” colorings of an underlying set of points relative to a given family of sets - is explored.
Journal ArticleDOI
Syntactic clustering of the Web
TL;DR: An efficient way to determine the syntactic similarity of files is developed and applied to every document on the World Wide Web, and a clustering of all the documents that are syntactically similar is built.
Journal ArticleDOI
Min-Wise Independent Permutations
TL;DR: This research was motivated by the fact that such a family of permutations is essential to the algorithm used in practice by the AltaVista web index software to detect and filter near-duplicate documents.
Proceedings Article
Finding similar files in a large file system
TL;DR: Application of sif can be found in file management, information collecting, program reuse, file synchronization, data compression, and maybe even plagiarism detection.
Proceedings ArticleDOI
Copy detection mechanisms for digital documents
TL;DR: This paper proposes a system for registering documents and then detecting copies, either complete copies or partial copies, and describes algorithms for such detection, and metrics required for evaluating detection mechanisms (covering accuracy, efficiency, and security).