Open Innovation: The New Imperative for Creating and Profiting from Technology

'Crowdsourcing' is a relatively recent concept that encompasses many practices. This diversity leads to the blurring of the limits of crowdsourcing that may be identified virtually with any type of internet-based collaborative activity, such as co-creation or user innovation. Varying definitions of crowdsourcing exist, and therefore some authors present certain specific examples of crowdsourcing as paradigmatic, while others present the same examples as the opposite. In this article, existing definitions of crowdsourcing are analysed to extract common elements and to establish the basic characteristics of any crowdsourcing initiative. Based on these existing definitions, an exhaustive and consistent definition for crowdsourcing is presented and contrasted in 11 cases.

/pdf/towards-an-integrated-crowdsourcing-definition-3qoy6yinnl.pdf

Towards an integrated crowdsourcing definition

A fundamental problem in many disciplines is the classification of objects in a domain of interest into a taxonomy. Developing a taxonomy, however, is a complex process that has not been adequately...

https://link.springer.com/content/pdf/10.1057%2Fejis.2012.26.pdf

A method for taxonomy development and its application in information systems

Most publicly available image quality databases have been created under highly controlled conditions by introducing graded simulated distortions onto high-quality photographs. However, images captured using typical real-world mobile camera devices are usually afflicted by complex mixtures of multiple distortions, which are not necessarily well-modeled by the synthetic distortions found in existing databases. The originators of existing legacy databases usually conducted human psychometric studies to obtain statistically meaningful sets of human opinion scores on images in a stringently controlled visual environment, resulting in small data collections relative to other kinds of image analysis databases. Toward overcoming these limitations, we designed and created a new database that we call the LIVE In the Wild Image Quality Challenge Database, which contains widely diverse authentic image distortions on a large number of images captured using a representative variety of modern mobile devices. We also designed and implemented a new online crowdsourcing system, which we have used to conduct a very large-scale, multi-month image quality assessment (IQA) subjective study. Our database consists of over 350 000 opinion scores on 1162 images evaluated by over 8100 unique human observers. Despite the lack of control over the experimental environments of the numerous study participants, we demonstrate excellent internal consistency of the subjective data set. We also evaluate several top-performing blind IQA algorithms on it and present insights on how the mixtures of distortions challenge both end users as well as automatic perceptual quality prediction models. The new database is available for public use at http://live.ece.utexas.edu/research/ChallengeDB/index.html .

Massive Online Crowdsourced Study of Subjective and Objective Picture Quality

Smartphones can reveal crowdsourcing's full potential and let users transparently contribute to complex and novel problem solving. This emerging area is illustrated through a taxonomy that classifies the mobile crowdsourcing field and through three new applications that optimize location-based search and similarity services based on crowd-generated data. Such applications can be deployed on SmartLab, a cloud of more than 40 Android devices deployed at the University of Cyprus that provides an open testbed to facilitate research and development of smartphone applications on a massive scale.

/pdf/crowdsourcing-with-smartphones-4qly37e484.pdf

Crowdsourcing with Smartphones

Crowdsourcing is an umbrella term for a variety of approaches that tap into the potential of a large and open crowd of people. So far, there is no systematic understanding of the processes used to source and aggregate contributions from the crowd. In particular, crowdsourcing organizations striving to achieve a specific goal should be able to evaluate the mechanisms that impact these processes. Following a method of IS taxonomy development we propose a new taxonomic framework for crowdsourcing processes. In contrast to previous work, this classification scheme focuses exclusively on an organizational perspective and on the mechanisms available to these organizations. The resulting dimensions are preselection of contributors, accessibility of peer contributions, aggregation of contributions, and remuneration for contributions. By classifying the processes of 46 crowdsourcing examples, we identify 19 distinct process types. A subsequent cluster analysis shows general patterns among these types and indicates a link to certain applications of crowdsourcing.

https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1396&context=amcis2011_submissions

Managing the Crowd: Towards a Taxonomy of Crowdsourcing Processes

Crowdsourcing information systems are socio-technical systems that provide informational products or services by harnessing the diverse potential of large groups of people via the Web. Interested individuals can contribute to such systems by selecting among a wide range of open tasks. Arguing that current approaches are suboptimal in terms of matching tasks and contributors' individual interests and capabilities, this article advocates the introduction of personalized task recommendation mechanisms. We contribute to a conceptual foundation for the design of such mechanisms by conducting a systematic review of the corresponding academic literature. Based on the insights derived from this analysis, we identify a number of issues for future research. In particular, our findings highlight the need for more significant empirical results through large-scale online experiments, an improved dialog with mainstream recommender systems research, and the integration of various sources of knowledge that exceed the boundaries of individual systems.

Personalized task recommendation in crowdsourcing information systems - Current state of the art

Crowdsourcing harnesses the potential of large and open networks of people. It is a relatively new phenomenon and attracted substantial interest in practice. Related research, however, lacks a theoretical foundation. We propose a system- theoretical perspective on crowdsourcing systems to address this gap and illustrate its applicability by using it to classify crowdsourcing systems. By deriving two principal dimensions from theory, we identify four fundamental types of crowdsourcing systems that help to distinguish important features of such systems. We analyse their respective characteristics and discuss implications and requirements for various aspects related to the design of such systems. Our results demonstrate that systems theory can inform the study of crowdsourcing systems. The identified system types and the implications on their design may prove useful for researchers to frame future studies and for practitioners to identify the right crowdsourcing systems for a particular purpose.

/pdf/crowdsourcing-information-systems-a-systems-theory-3tracahl5q.pdf

Crowdsourcing Information Systems - A Systems Theory Perspective

In the last years, crowdsourcing has emerged as a new approach for outsourcing work to a large number of human workers in the form of an open call. Amazon’s Mechanical Turk (MTurk) enables requesters to efficiently distribute micro tasks to an unknown workforce which selects and processes them for small financial rewards. While worker behavior and demographics as well as task design and quality management have been studied in detail, more research is needed on the relationship between workers and task design. In this paper, we conduct a series of explorative studies on task properties on MTurk. First, we identify properties that may be relevant to workers’ task selection through qualitative and quantitative preliminary studies. Second, we provide a quantitative survey with 345 participants. As a result, the task properties are ranked and set into relation with the workers’ demographics and background. The analysis suggests that there is little influence of education level, age, and gender. Culture may influence the importance of bonuses, however. Based on the explorative data analysis, five hypotheses for future research are derived. This paper contributes to a better understanding of task choice and implies that other factors than demographics influence workers’ task selection.

Exploring task properties in crowdsourcing – an empirical study on mechanical turk

Crowdsourcing has become a popular approach for capitalizing on the potential of large and open crowds of people external to the organization. While crowdsourcing as a phenomenon is studied in a variety of fields, research mostly focuses on isolated aspects and little is known about the integrated design of crowdsourcing efforts. We introduce a socio-technical systems perspective on crowdsourcing, which provides a deeper understanding of the components and relationships in crowdsourcing systems. By considering the function of crowdsourcing systems within their organizational context, we develop a typology of four distinct system archetypes. We analyze the characteristics of each type and derive a number of design requirements for the respective system components. The paper lays a foundation for IS-based crowdsourcing research, channels related academic work, and helps guiding the study and design of crowdsourcing information systems.

David Geiger

Papers

Managing the Crowd: Towards a Taxonomy of Crowdsourcing Processes

Personalized task recommendation in crowdsourcing information systems - Current state of the art

Crowdsourcing Information Systems - A Systems Theory Perspective

Exploring task properties in crowdsourcing – an empirical study on mechanical turk

Crowdsourcing Information Systems - Definition, Typology, and Design