Quantitative evaluation of the effectiveness of idea generation in the wild

doi:10.1007/978-3-642-21753-1_14

Home
/
Papers
/
Quantitative evaluation of the effectiveness of idea generation in the wild

Book Chapter•DOI•

Quantitative evaluation of the effectiveness of idea generation in the wild

Lassi A. Liikkanen¹, Matti M. Hämäläinen², Anders Häggman³, Tua Björklund², Mikko P. Koskinen² - Show less +1 more•Institutions (3)

Helsinki Institute for Information Technology¹, Aalto University², Massachusetts Institute of Technology³

09 Jul 2011-pp 120-129

TL;DR: An experimental, quantitative methodology from the domain of product design research for evaluating different idea generation methods is described and prominent results from relevant literature and new data from a study of idea generation in the wild are presented.

read less

Abstract: New ideas are the primary building blocks in attempts to produce novel interactive technology. Numerous idea generation methods such as Brainstorming have been introduced to support this process, but there is mixed evidence regarding their effectiveness. In this paper we describe an experimental, quantitative methodology from the domain of product design research for evaluating different idea generation methods. We present prominent results from relevant literature and new data from a study of idea generation in the wild. The study focused on the effects of the physical environment, or in other words, the physical context, on designers' capacity to produce ideas. 25 students working in small groups took part in an experiment with two design tasks. Moving from an office environment to the actual surroundings of the intended use, we discovered that the change in resulting ideas was surprisingly small. Of the measured dimensions, the real-world context influenced only the feasibility of ideas, leaving quantity, novelty, utility and level of detail unaffected. This finding questions the value of diving into the context as a design idea generation practice.

...read moreread less

Citations

PDF

Open Access

More filters

Journal Article•DOI•

Improving the idea selection process in creative workshops through contextualisation

[...]

Alex Gabriel¹, Mauricio Camargo¹, Davy Monticolo¹, Vincent Boly¹, Mario Bourgault² - Show less +1 more•Institutions (2)

University of Lorraine¹, École Polytechnique de Montréal²

01 Nov 2016-Journal of Cleaner Production

TL;DR: A context formalisation approach is proposed, supported by multi-criteria decision-making analysis tools to evaluate ideas resulting from a creative workshop, that seeks to assist the decision- making by requiring success criteria to take into account the firm's context and priorities and give decision-makers the keys to establish a coherent evaluation.

...read moreread less

23 citations

A Compendium of Evidence for Creative Problem Solving

[...]

Scott G. Isaksen

01 Jan 2015

22 citations

Dissertation•

Gestion des connaissances lors d’un processus collaboratif de créativité

[...]

Alex Gabriel

07 Dec 2016

TL;DR: L’architecture multi-agents proposee pour concevoir un systeme support a la creativite permet d’explorer de nouveaux modes de traitement des connaissances notamment concernant l’evaluation des idees.

...read moreread less

Abstract: Le travail de cette these vise a proposer un systeme support a la creativite selon une architecture multi-agents afin de gerer les connaissances necessaires et produites durant un atelier de creativite. Ce travail contribue a la recherche scientifique a differents egards. Au prealable de concevoir un quelconque systeme, une revue des systemes actuels supportant la creativite est realisee pour determiner leurs limites en termes de processus de creativite et de modes de collaboration. Pour repondre a ces limites, l’approche d’ingenierie des connaissances est adoptee. Autrement dit, la creativite est consideree comme un processus collaboratif et organisationnel afin d’en deduire les activites a supporter ainsi que les connaissances et les competences a apporter aux individus pour les assister a realiser ces activites. A partir de la modelisation organisationnelle d’un atelier de creativite, l’organisation des agents informatiques qui vont contribuer a la gestion des connaissances en est deduite. Par la suite, une ontologie de l’atelier de creativite est formalisee a partir de la modelisation de l’organisation afin d’apporter une representation des connaissances et de l’environnement aux agents. Ainsi, l’architecture multi-agents proposee pour concevoir un systeme support a la creativite permet d’explorer de nouveaux modes de traitement des connaissances notamment concernant l’evaluation des idees. Une methodologie d’evaluation des idees selon des methodes d’analyse multicritere est proposee. En complement de cette methodologie, le traitement automatique des idees a ete experimente afin d’aider les evaluateurs dans leur tâche

...read moreread less

15 citations

Additional excerpts

...…à dents Créativité, Nouveauté, Utilité, Louabilité, Clarté (Gray et al., 2010) Artefact Divers conceptions d’objets Nouveauté, Utilité, Faisabilité (Liikkanen et al., 2011) Situationnel Réception d’hôtel informatisée et Smart Gym Nouveauté, Utilité, Faisabilité, Niveau de détail, Mode de…...
[...]
...Sélection d’études concernant l’évaluation des idées et les critères utilisés Etude Nature de l’étude Produit / situation Critères impliqués (Dean et al. 2006) Situationnel Augmenter le tourisme à Tuscon Nouveauté, Faisabilité, Pertinence, Spécificité (Kudrowitz and Wallace, 2013) Artefact Parapluies, grille-pains et brosses à dents Créativité, Nouveauté, Utilité, Louabilité, Clarté (Gray et al., 2010) Artefact Divers conceptions d’objets Nouveauté, Utilité, Faisabilité (Liikkanen et al., 2011) Situationnel Réception d’hôtel informatisée et Smart Gym Nouveauté, Utilité, Faisabilité, Niveau de détail, Mode de présentation (Jagtap et al., 2015) Artefact Divers conceptions d’objets Qualité, Nouveauté, Variété (Cluzel et al., 2016) Situationnel Sélectionner des projets de R&D Eco-Innovant Nouveauté, Variété, Quantité, Qualité (Correa and Danilevicz, 2015) Situationnel Sélectionner des idées de projets innovants pour l’industrie électrique Alignement à la stratégie de l’entreprise, Faisabilité, Retours financier, social et environnemental (Verhaegen et al., 2013) Artefact Rasoirs Quantité, Variété, Nouveauté, Qualité Formalisation du contexte 7.1.2....
[...]

Book Chapter•DOI•

Computational Design Creativity Evaluation

[...]

David C. Brown¹•Institutions (1)

Worcester Polytechnic Institute¹

01 Jan 2015

TL;DR: In this paper, the authors present a simple framework for computational design creativity evaluation, presenting its components with rationale, which are linked to recent computational creativity research in both art and design, and that evaluation is done by comparison with descriptions of existing products using a set of aspects that each suggest creativity.

...read moreread less

Abstract: This paper presents a simple framework for computational design creativity evaluation, presenting its components with rationale. Components are linked to recent computational creativity research in both art and design. The framework assumes that the product, not the process, is being evaluated, and that evaluation is done by comparison with descriptions of existing products using a set of aspects that each suggest creativity. Not every evaluation will use all of the components of the framework. It can be used to guide or assess design creativity research.

...read moreread less

11 citations

Geo-spatial Mapping as a Catalyst for Creative and Engaged Design in Engineering Education

[...]

jessie zarazaga

01 Jan 2019

TL;DR: In this paper, the authors present a survey of the educational assessment, evaluation, and research commons, including the following: Educational Methods Commons, Environmental Design Commons, Landscape Architecture Commons, Other Civil and Environmental Engineering Commons, Scholarship of Teaching and Learning Commons, Sustainability Commons, and the Urban, Community and Regional Planning Commons.

...read moreread less

Abstract: Follow this and additional works at: https://scholar.smu.edu/engineering_multidisciplinary_etds Part of the Educational Assessment, Evaluation, and Research Commons, Educational Methods Commons, Environmental Design Commons, Landscape Architecture Commons, Other Civil and Environmental Engineering Commons, Scholarship of Teaching and Learning Commons, Sustainability Commons, and the Urban, Community and Regional Planning Commons

...read moreread less

10 citations

Cites methods from "Quantitative evaluation of the effe..."

...similarly developed five quantifiable variables for experimental conditions: along with Quantity and Novelty, they assessed Feasibility against the diversity of results in the pool, and Quality, once again, by independent experts within the domain (Liikkanen et al., 2011)....
[...]

1
2
3
4
…

References

PDF

Open Access

More filters

Book Chapter•DOI•

Handbook of creativity

[...]

Robert J. Sternberg

01 Jan 1998

TL;DR: A history of research on creativity can be found in this paper, where a case study method and evolving systems approach for understanding unique creative people at work is presented. But it is not a comprehensive survey of all the works on creativity.

...read moreread less

Abstract: Part I. Introduction: 1. The concept of creativity: prospects and paradigms Robert J. Sternberg and Todd Lubart 2. A history of research on creativity Robert S. Albert and Mark Runco Part II. Methods for Studying Creativity: 3. Psychometric approaches to the study of human creativity Jonathan A. Plucker and Joseph Renzulli 4. Experimental studies of creativity Mark Runco and Shawn Okuda Sakamoto 5. The case study method and evolving systems approach for understanding unique creative people at work Howard E. Gruber and Doris Wallace 6. Creativity from a historiometric perspective Dean Keith Simonton Part III. Origins of Creativity: 7. Biological bases of creativity Colin Martindale 8. Evolving creative minds: stories and mechanisms Charles J. Lumsden 9. The development of creativity David Henry Feldman Part IV. Creativity, the Self and Environment: 10. Creative cognition Thomas B. Ward, Steven M. Smith and Ronald A. Finke 11. From case studies to robust generalizations: an approach to the study of creativity Emma Policastro and Howard Gardner 12. Creativity and knowledge: a challenge to theories Robert W. Weisberg 13. Creativity and intelligence Robert W. Weisberg and Linda O'Hara 14. The influence of personality on artistic and scientific creativity Gregory J. Feist 15. Motivation and creativity Mary Ann Collins and Teresa Amabile 16. Implications of a systems perspective for the study of creativity Mihaly Csikszentmihalyi Part V. Special Topics in Creativity: 17. Creativity across cultures Todd I. Lubart 18. Computer models of creativity Margaret A. Boden 19. Organizational creativity Wendy M. Williams and Lana T. Yang 20. Enhancing creativity Raymond S. Nickerson 21. Prodigies and creativity Michael J. A. Howe Part VI. Conclusion: 22. Fifty years of creativity research Richard E. Mayer.

...read moreread less

2,451 citations

Journal Article•DOI•

Productivity loss in brainstorming groups: Toward the solution of a riddle.

[...]

Michael Diehl, Wolfgang Stroebe

01 Sep 1987-Journal of Personality and Social Psychology

TL;DR: In this paper, the authors investigate free riding, evaluation apprehension, and production blocking as explanations of the difference in brainstorming productivity typically observed between real and nominal groups and conclude that production blocking accounted for most of the productivity loss of real brainstorming groups.

...read moreread less

Abstract: We conducted four experiments to investigate free riding, evaluation apprehension, and production blocking as explanations of the difference in brainstorming productivity typically observed between real and nominal groups. In Experiment 1, we manipulated assessment expectations in group and individual brainstorming. Although productivity was higher when subjects worked under personal rather than collective assessment instructions, type of session still had a major impact on brainstorming productivity under conditions that eliminated the temptation to free ride. Experiment 2 demonstrated that inducing evaluation apprehension reduced productivity in individual brainstorming. However, the failure to find an interaction between evaluation apprehension and type of session in Experiment 3 raises doubts about evaluation apprehension as a major explanation of the productivity loss in brainstorming groups. Finally, by manipulating blocking directly, we determined in Experiment 4 that production blocking accounted for most of the productivity loss of real brainstorming groups. The processes underlying production blocking are discussed, and a motivational interpretation of blocking is offered. In his influential book, Osborn (1957) suggested brainstorming as a method of group problem solving that considerably increases the quality and quantity of ideas produced by group members. Brainstorming groups are traditionally given instructions designed to free the individual members from the inhibiting effects of self-criticism and the criticism by others during the problem-solving session. The rules behind brainstorming are as follows: keep in mind that the more ideas the better and the wilder the ideas the better, improve or combine ideas already suggested; and do not be critical. Osborn (1957) claimed that if these rules are followed "the average person can think up twice as many ideas when working with a group than when working alone" (p. 229).

...read moreread less

1,808 citations

"Quantitative evaluation of the effe..." refers background in this paper

...Group situations generally induce phenomena known as production blocking, free riding, evaluation apprehension [3], and different forms of group think....
[...]
...It has been repeatedly shown that real groups of more than two people always underperform in comparison to nominal groups [3]....
[...]

Book•

Applied imagination : principles and procedures of creative problem-solving /

[...]

Alexander Faickney Osborn

01 Jan 1963

1,319 citations

"Quantitative evaluation of the effe..." refers methods in this paper

...In the tradition of this craft, formal IG methods such as synectics, six thinking hats, morphological analysis, lateral thinking, Quantitative Evaluation of the Effectiveness of IG in the Wild 121 TRIZ, and naturally Brainstorming [1], can be found....
[...]
...In the tradition of this craft, formal IG methods such as synectics, six thinking hats, morphological analysis, lateral thinking, TRIZ, and naturally Brainstorming [1], can be found....
[...]

Journal Article•DOI•

Metrics for measuring ideation effectiveness

[...]

Jami J. Shah¹, Steve M. Smith², Noe Vargas-Hernandez¹•Institutions (2)

Arizona State University¹, Texas A&M University²

01 Mar 2003-Design Studies

TL;DR: In this paper, the authors describe four objective measures of ideation effectiveness, and the theoretical basis of each is discussed and procedures for application of each are outlined and illustrated with case studies.

...read moreread less

977 citations

"Quantitative evaluation of the effe..." refers background in this paper

...They most often concern product design and have been carried out in the engineering domain [12-14]....
[...]

Journal Article•DOI•

Expertise in Design: an overview

[...]

Nigel Cross¹•Institutions (1)

Open University¹

01 Sep 2004-Design Studies

TL;DR: A review of the field of research in expertise in design can be found in this paper, where the authors focus on expert performance in design and design expertise in other fields, and it seems that expertise has some aspects that are significantly different from expertise from other fields.

...read moreread less

907 citations