Proceedings ArticleDOI
On the resemblance and containment of documents
TLDR
The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that could be done independently for each document.Abstract:
Given two documents A and B we define two mathematical notions: their resemblance r(A, B) and their containment c(A, B) that seem to capture well the informal notions of "roughly the same" and "roughly contained." The basic idea is to reduce these issues to set intersection problems that can be easily evaluated by a process of random sampling that can be done independently for each document. Furthermore, the resemblance can be evaluated using a fixed size sample for each document. This paper discusses the mathematical properties of these measures and the efficient implementation of the sampling process using Rabin (1981) fingerprints.read more
Citations
More filters
Proceedings ArticleDOI
Domino Effect: Move More Methods Once a Method is Moved
TL;DR: A new way to identify methods that should be moved is proposed that checks other methods within the same class, and suggests to move the one with the greatest similarity and strongest relationship with the moved method.
Proceedings Article
Longer is Better? A Case Study of Product Review Helpfulness Prediction
TL;DR: This research investigates the factors that moderate the impact of review length on review helpfulness, and explores the interaction between review length and several other factors, including readability, the number of product features, and redundancy of products features.
Proceedings ArticleDOI
Distance-Sensitive Hashing
TL;DR: This paper begins the study of distance-sensitive hashing (DSH), a generalization of LSH that seeks a family of hash functions such that the probability of two points having the same hash value is a given function of the distance between them, and extends existing LSH lower bounds, showing that they also hold in the asymmetric setting.
Journal ArticleDOI
Pigeonring: A Principle for Faster Thresholded Similarity Search
Jianbin Qin,Chuan Xiao +1 more
TL;DR: A universal filtering framework is introduced to encompass the solutions to problems defined in the form of identifying data objects whose similarities or distances to the query is constrained by a threshold and shows that the pigeonhole principle is a special case of the new principle.
Book ChapterDOI
Increasing recall for text re-use in historical documents to support research in the humanities
TL;DR: Using an edition of one Greek work that marked quotations and paraphrases from the Homeric epics as a testbed, this paper was able to achieve a recall of at least 94% while maintaining a precision of 73%.
References
More filters
Book
The Probabilistic Method
TL;DR: A particular set of problems - all dealing with “good” colorings of an underlying set of points relative to a given family of sets - is explored.
Journal ArticleDOI
Syntactic clustering of the Web
TL;DR: An efficient way to determine the syntactic similarity of files is developed and applied to every document on the World Wide Web, and a clustering of all the documents that are syntactically similar is built.
Journal ArticleDOI
Min-Wise Independent Permutations
TL;DR: This research was motivated by the fact that such a family of permutations is essential to the algorithm used in practice by the AltaVista web index software to detect and filter near-duplicate documents.
Proceedings Article
Finding similar files in a large file system
TL;DR: Application of sif can be found in file management, information collecting, program reuse, file synchronization, data compression, and maybe even plagiarism detection.
Proceedings ArticleDOI
Copy detection mechanisms for digital documents
TL;DR: This paper proposes a system for registering documents and then detecting copies, either complete copies or partial copies, and describes algorithms for such detection, and metrics required for evaluating detection mechanisms (covering accuracy, efficiency, and security).