Paid crowd work offers remarkable opportunities for improving productivity, social mobility, and the global economy by engaging a geographically distributed workforce to complete complex tasks on demand and at scale. But it is also possible that crowd work will fail to achieve its potential, focusing on assembly-line piecework. Can we foresee a future crowd workplace in which we would want our children to participate? This paper frames the major challenges that stand in the way of this goal. Drawing on theory from organizational behavior and distributed computing, as well as direct feedback from workers, we outline a framework that will enable crowd work that is complex, collaborative, and sustainable. The framework lays out research challenges in twelve major areas: workflow, task assignment, hierarchy, real-time response, synchronous collaboration, quality control, crowds guiding AIs, AIs guiding crowds, platforms, job design, reputation, and motivation.

/pdf/the-future-of-crowd-work-2houizqbg3.pdf

The future of crowd work

The Future of Crowd Work

The so-called “gig-economy” has been growing exponentially in numbers and importance in recent years but its impact on labour rights has been largely overlooked. Forms of work in the “gig-economy” include “crowdwork”, and “work-on-demand via apps”, under which the demand and supply of working activities is matched online or via mobile apps. These forms of work can provide a good match of job opportunities and allow flexible working schedules. However, they can also pave the way to a severe commodification of work. This paper discusses the implications of this commodification and advocates the full recognition of activities in the gig-economy as “work”.

/pdf/the-rise-of-the-just-in-time-workforce-on-demand-work-3z74p8d84v.pdf

The rise of the 'just-in-time workforce': On-demand work, crowdwork and labour protection in the 'gig-economy'

Why peer discussion improves student performance on in-class concept questions

As academic and practitioner studies on crowdsourcing have been building up since 2006, the subject itself has progressively gained in importance within the broad field of management. No systematic review on the topic has so far appeared in management journals, however; moreover, the field suffers from ambiguity in the topic's definition, which in turn has led to its largely unstructured evolution. The authors therefore investigate the existing body of knowledge on crowdsourcing systematically through a penetrating review in which the strengths and weakness of this literature stream are presented clearly and then future avenues of research are set out. The review is based on 121 scientific articles published between January 2006 and January 2015. The review recognizes that crowdsourcing is ingrained in two mainstream disciplines within the broader subject matter of innovation and management: (1) open innovation; and (2) co-creation. The review, in addition, also touches on several issues covered in other theoretical streams: (3) information systems management; (4) organizational theory and design; (5) marketing; and (6) strategy. The authors adopt a process perspective, applying the ‘Input–Process–Output’ framework to interpret research on crowdsourcing within the broad lines of: (1) Input (Problem/Task); (2) Process (session management; problem management; knowledge management; technology); and (3) Outcome (solution/completed task; seekers’ benefits; solvers’ benefits). This framework provides a detailed description of how the topic has evolved over time, and suggestions concerning the future direction of research are proposed in the form of research questions that are valuable for both academics and managers.

/pdf/crowdsourcing-a-review-and-suggestions-for-future-research-jn2tv5mkqp.pdf

Crowdsourcing: A Review and Suggestions for Future Research

Existing crowdsourcing markets are often inaccessible to workers living at the bottom of the economic pyramid. We present MobileWorks, a mobile phone-based crowdsourcing platform intended to provide employment to developing world users. MobileWorks provides human optical character recognition (OCR) tasks that can be completed by workers on low-end mobile phones through a web browser. To address the limited screen resolution available on low-end phones, MobileWorks divides documents into many small pieces and sends each piece to a different worker. An initial pilot study with 10 users over a two month period revealed that it is feasible to do basic OCR tasks using a simple mobile web-based application. We find that workers using Mobile Works average 120 tasks per hour at an accuracy rate of 99% using a multiple entry solution. In addition, users had a positive experience with MobileWorks: all study participants would recommend MobileWorks to friends and family.

/pdf/mobileworks-a-mobile-crowdsourcing-platform-for-workers-at-3tu4jh0rk2.pdf

MobileWorks: a mobile crowdsourcing platform for workers at the bottom of the pyramid

Online labor marketplaces offer the potential to automate tasks too difficult for computers, but don't always provide accurate results. MobileWorks is a crowd platform that departs from the marketplace model to provide robust, high-accuracy results using three new techniques. A dynamic work-routing system identifies expertise in the crowd and ensures that posted work completes within a bounded time with fair wages. A peer-management system helps prevent wrong answers. Last, social interaction techniques let best workers manage and teach other crowd members. This process allows the crowd to collaboratively learn how to solve new tasks.

MobileWorks: Designing for Quality in a Managed Crowdsourcing Architecture

Remotely instructing and guiding users in physical tasks has offered promise across a wide variety of domains. While it has been the subject of many research projects, current approaches are often limited in the communication bandwidth (lacking context, spatial information) or interactivity (unidirectional, asynchronous) between the expert and the learner. Systems that use Mixed-Reality systems for this purpose have rigid configurations for the expert and the learner. We explore the design space of bi-directional mixed-reality telepresence systems for teaching physical tasks, and present Loki, a novel system which explores the various dimensions of this space. Loki leverages video, audio and spatial capture along with mixed-reality presentation methods to allow users to explore and annotate the local and remote environments, and record and review their own performance as well as their peer's. The system design of Loki also enables easy transitions between different configurations within the explored design space. We validate its utility through a varied set of scenarios and a qualitative user study.

Loki: Facilitating Remote Instruction of Physical Tasks Using Bi-Directional Mixed-Reality Telepresence

MobileWorks: Designing for Quality in a Managed Crowdsourcing Architecture (Extended Abstract).

Amateur instructional videos often show a single uninterrupted take of a recorded demonstration without any edits. While easy to produce, such videos are often too long as they include unnecessary or repetitive actions as well as mistakes. We introduce DemoCut, a semi-automatic video editing system that improves the quality of amateur instructional videos for physical tasks. DemoCut asks users to mark key moments in a recorded demonstration using a set of marker types derived from our formative study. Based on these markers, the system uses audio and video analysis to automatically organize the video into meaningful segments and apply appropriate video editing effects. To understand the effectiveness of DemoCut, we report a technical evaluation of seven video tutorials created with DemoCut. In a separate user evaluation, all eight participants successfully created a complete tutorial with a variety of video editing effects using our system.

Bjoern Hartmann

Papers

MobileWorks: a mobile crowdsourcing platform for workers at the bottom of the pyramid

MobileWorks: Designing for Quality in a Managed Crowdsourcing Architecture

Loki: Facilitating Remote Instruction of Physical Tasks Using Bi-Directional Mixed-Reality Telepresence

MobileWorks: Designing for Quality in a Managed Crowdsourcing Architecture (Extended Abstract).

DemoCut: generating concise instructional videos for physical demonstrations