In this paper, we present Google, a prototype of a large-scale search engine which makes heavy use of the structure present in hypertext. Google is designed to crawl and index the Web efficiently and produce much more satisfying search results than existing systems. The prototype with a full text and hyperlink database of at least 24 million pages is available at http://google.stanford.edu/. To engineer a search engine is a challenging task. Search engines index tens to hundreds of millions of web pages involving a comparable number of distinct terms. They answer tens of millions of queries every day. Despite the importance of large-scale search engines on the web, very little academic research has been done on them. Furthermore, due to rapid advance in technology and web proliferation, creating a web search engine today is very different from three years ago. This paper provides an in-depth description of our large-scale web search engine -- the first such detailed public description we know of to date. Apart from the problems of scaling traditional search techniques to data of this magnitude, there are new technical challenges involved with using the additional information present in hypertext to produce better search results. This paper addresses this question of how to build a practical large-scale system which can exploit the additional information present in hypertext. Also we look at the problem of how to effectively deal with uncontrolled hypertext collections where anyone can publish anything they want.

/pdf/the-anatomy-of-a-large-scale-hypertextual-web-search-engine-496poj789d.pdf

The anatomy of a large-scale hypertextual Web search engine

Imagined communities: Reflections on the origin and spread of nationalism

The Anatomy of a Large-Scale Hypertextual Web Search Engine.

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

The development of the Internet in recent years has made it possible and useful to access many different information systems anywhere in the world to obtain information. While there is much research on the integration of heterogeneous information systems, most commercial systems stop short of the actual integration of available data. Data fusion is the process of fusing multiple records representing the same real-world object into a single, consistent, and clean representation.This article places data fusion into the greater context of data integration, precisely defines the goals of data fusion, namely, complete, concise, and consistent data, and highlights the challenges of data fusion, namely, uncertain and conflicting data values. We give an overview and classification of different ways of fusing data and present several techniques based on standard and advanced operators of the relational algebra and SQL. Finally, the article features a comprehensive survey of data integration systems from academia and industry, showing if and how data fusion is performed in each.

Data fusion

Experiences of contingent responsivity during shared book reading predict better learning outcomes However, it is unclear whether contingent responsivity from a digital book could provide similar support for children The effects on story recall and engagement interacting with a digital book that responded contingently on children's vocalizations (contingent book) were investigated, with a focus on the role of individual differences in attention The study used a within-subject design with 3 experiments from 90 3- to 5-year-old children Children were presented with a contingent book and 3 noncontingent control conditions: a board book (Experiment 1), a static digital book (Experiment 2), and an animated book (Experiment 3) The use of the contingent book significantly increased children's story recall and was found to be especially useful for children with less developed attention regulation (PsycINFO Database Record (c) 2020 APA, all rights reserved)

Contingent responsivity in E-books modeled from quality adult-child interactions: Effects on children's learning and attention.

A key challenge for mixed-initiative systems is to create a shared understanding of the task between human and agent To address this challenge, we created a mixed-initiative interface called Mixer to aid administrators with automating tedious information-retrieval tasks Users initiate communication with the agent by constructing a form, creating a structure to hold the information they require and to show context in order to interpret this information They then populate the form with the desired results, demonstrating to the agent the steps required to retrieve the information This method of form creation explicitly defines the shared understanding between human and agent An evaluation of the interface shows that administrators can effectively create forms to communicate with the agent, that they are likely to accept this technology in their work environment, and that the agent's help can significantly reduce the time they spend on repeated information-retrieval tasks

/pdf/user-created-forms-as-an-effective-method-of-human-agent-1a6eoz6tf6.pdf

User-created forms as an effective method of human-agent communication

The scientific community, public organizations and administrations have generated a large amount of data concerning the environment. There is a need to allow sharing and exchange of this type of information by various kinds of users including scientists, decision-makers and public authorities. Metadata arises as the solution to support these requirements. We present a formal framework for classification of metadata that will give a uniform definition of what metadata is, how it can be used and where it must be used. This framework also provides a procedure for classifying elements of existing metadata standards.

A Framework for Classifying Scientific Metadata

Mediator systems are used today in a wide variety of unreliable environments. When processing a query, a mediator may try to access a data source which is unavailable. In this situation, existing systems either silently ignore unavailable data sources or generate an error. In either case, to obtain the complete answer, the query is reprocessed from scratch. This behavior is inefficient in environments with a non-negligible probability that a data source is unavailable (e.g., the Internet). In the case that some data sources are unavailable, the complete answer to a query cannot be obtained; however useful work can be done with the available data sources. In this paper, we describe a novel approach to mediator query processing where, in the presence of unavailable data sources, the answer to a query is a {\em partial answer}. The partial answer represents the state of the mediator at the end of query processing, i.e., materialized data. This state is used to construct an {\em incremental query}. The answer to the incremental query is the same as the complete answer, but it is more efficient to evaluate than the original query. In addition, information can be extracted from the mediator state through the use of secondary queries, called {\em parachute queries}. We describe an intuitive class of parachute queries and an algorithm which generates it.We define two new architectures for partial answers, incremental and parachute queries and analytically model for these architectures the probability of obtaining the answer to query in the presence of unavailable data sources. The analysis shows that complete answers are more likely in our two architectures than in a classical system. We measure the performance of our architectures via simulations and show that, in the case that all data sources are available, the performance penalty for our approach is negligible. In addition, we show that there is a trade-off between the cost of query execution and the probability of obtaining a complete or parachute query answer.

https://hal.inria.fr/inria-00073261/document

Parachute Queries in the Presence of Unavailable Data Sources

Crowdsourced mobile sensing systems provide a counterpoint to the idea of fully automated sensing systems by transferring some or all of the sensing duties to the end users. Humans can easily sense in some ways that are impossible for machines to sense, leading to hybrid crowdsourced-automated systems. However, this transfer of sensing to humans comes with design trade-offs in terms of the sparsity of the sensed information and human entry errors. To better understand these design trade-offs, the authors developed a real-time arrival information system for a local transit agency that crowdsources the location of transit vehicles by having riders share location traces from their smart phones. The authors then deployed the system and measured the public’s use of it for 10 months, gathering data on 296,283 interactive sessions. Analysis shows that relying on users can produce very sparse information for the whole system but much better information relative to the times and places that users access the service. Reflecting on this deployment, the authors conclude that crowdsourced-sensing systems perform well when (i) the value of an observation persists over time, (ii) many people make observations, and (iii) observations would be difficult or expensive to sense with an automated sensing system.

Anthony Tomasic

Papers

Contingent responsivity in E-books modeled from quality adult-child interactions: Effects on children's learning and attention.

User-created forms as an effective method of human-agent communication

A Framework for Classifying Scientific Metadata

Parachute Queries in the Presence of Unavailable Data Sources

The Performance of a Crowdsourced Transportation Information System