Showing papers by "Jeffrey Dean published in 2010"

MapReduce: a flexible data processing tool

Abstract: MapReduce advantages over parallel databases include storage-system independence and fine-grain fault tolerance for large jobs.

System and method for large-scale data processing using an application-independent framework

Abstract: A large-scale data processing system and method for processing data in a distributed and parallel processing environment The system includes an application-independent framework for processing data having a plurality of application-independent map modules and reduce modules These application-independent modules use application-independent operators to automatically handle parallelization of computations across the distributed and parallel processing environment when performing user-specified data processing operations The system also includes a plurality of user-specified, application-specific operators, for use with the application-independent framework to perform a user-specified data processing operation on a user-specified set of input files The application-specific operators include: a map operator and a reduce operator The map operator is applied by the application-independent map modules to input data in the user-specified set of input files to produce intermediate data values The reduce operator is applied by the application-independent reduce modules to process the intermediate data values to produce final output data

Document scoring based on link-based criteria

Abstract: A method may include receiving a document and an initial score for the document; determining that there has been a decrease in a rate or quantity of new links that point to the document over time; classifying the document as stale in response to the determining; decreasing the initial score for the document, resulting in an updated score; and ranking the document with regard to at least one other document based, at least in part, on the score.

Document scoring based on document inception date

Abstract: A system may determine a document inception date associated with a document, generate a score for the document based, at least in part, on the document inception date, and rank the document with regard to at least one other document based, at least in part, on the score.