Proceedings ArticleDOI
CrowdDB: answering queries with crowdsourcing
Michael J. Franklin,Donald Kossmann,Tim Kraska,Sukriti Ramesh,Reynold Xin +4 more
- pp 61-72
Reads0
Chats0
TLDR
The design of CrowdDB is described, a major change is that the traditional closed-world assumption for query processing does not hold for human input, and important avenues for future work in the development of crowdsourced query processing systems are outlined.Abstract:
Some queries cannot be answered by machines only. Processing such queries requires human input for providing information that is missing from the database, for performing computationally difficult functions, and for matching, ranking, or aggregating results based on fuzzy criteria. CrowdDB uses human input via crowdsourcing to process queries that neither database systems nor search engines can adequately answer. It uses SQL both as a language for posing complex queries and as a way to model data. While CrowdDB leverages many aspects of traditional database systems, there are also important differences. Conceptually, a major change is that the traditional closed-world assumption for query processing does not hold for human input. From an implementation perspective, human-oriented query operators are needed to solicit, integrate and cleanse crowdsourced data. Furthermore, performance and cost depend on a number of new factors including worker affinity, training, fatigue, motivation and location. We describe the design of CrowdDB, report on an initial set of experiments using Amazon Mechanical Turk, and outline important avenues for future work in the development of crowdsourced query processing systems.read more
Citations
More filters
Journal ArticleDOI
Crowdsourcing systems on the World-Wide Web
TL;DR: The practice of crowdsourcing is transforming the Web and giving rise to a new field of inquiry called "crowdsourcing", which aims to provide real-time information about events in a democratic manner.
Proceedings ArticleDOI
The future of crowd work
Aniket Kittur,Jeffrey V. Nickerson,Michael S. Bernstein,Elizabeth M. Gerber,Aaron Shaw,John Zimmerman,Matthew Lease,John Horton +7 more
TL;DR: This paper outlines a framework that will enable crowd work that is complex, collaborative, and sustainable, and lays out research challenges in twelve major areas: workflow, task assignment, hierarchy, real-time response, synchronous collaboration, quality control, crowds guiding AIs, AIs guiding crowds, platforms, job design, reputation, and motivation.
Posted Content
The Future of Crowd Work
Aniket Kittur,Jeffrey V. Nickerson,Michael S. Bernstein,Elizabeth M. Gerber,Aaron Shaw,Aaron Shaw,John Zimmerman,Matthew Lease,John Horton +8 more
TL;DR: In this paper, the authors outline a framework that will enable crowd work that is complex, collaborative, and sustainable, and lay out research challenges in twelve major areas: workflow, task assignment, hierarchy, real-time response, synchronous collaboration, quality control, crowds guiding AIs, AIs guiding crowds, platforms, job design, reputation, and motivation.
Journal ArticleDOI
Evaluation on crowdsourcing research: Current status and future direction
Yuxiang Zhao,Qinghua Zhu +1 more
TL;DR: A critical examination of the substrate of crowdsourcing research is presented by surveying the landscape of existing studies, including theoretical foundations, research methods, and research foci, and identifies several important research directions for IS scholars from three perspectives—the participant, organization, and system—and which warrant further study.
Journal ArticleDOI
CrowdER: crowdsourcing entity resolution
TL;DR: This work proposes a hybrid human-machine approach in which machines are used to do an initial, coarse pass over all the data, and people are use to verify only the most likely matching pairs, and develops a novel two-tiered heuristic approach for creating batched tasks.
References
More filters
Book
Introduction to Information Retrieval
TL;DR: In this article, the authors present an up-to-date treatment of all aspects of the design and implementation of systems for gathering, indexing, and searching documents; methods for evaluating systems; and an introduction to the use of machine learning methods on text collections.
Proceedings ArticleDOI
Labeling images with a computer game
Luis von Ahn,Laura Dabbish +1 more
TL;DR: A new interactive system: a game that is fun and can be used to create valuable output that addresses the image-labeling problem and encourages people to do the work by taking advantage of their desire to be entertained.
Book
MDA Explained: The Model Driven Architecture¿: Practice and Promise
TL;DR: Insight is given in what MDA means and what you can achieve, both today and in the future, thereby raising the level of maturity of the IT industry.
Book
Database Management Systems
TL;DR: New to this edition are the early coverage of the ER model, new chapters on Internet databases, data mining, and spatial databases, and a new supplement on practical SQL assignments (with solutions for instructors' use).
Proceedings ArticleDOI
Who are the crowdworkers?: shifting demographics in mechanical turk
TL;DR: How the worker population has changed over time is described, shifting from a primarily moderate-income, U.S. based workforce towards an increasingly international group with a significant population of young, well-educated Indian workers.