Toward a Conceptual Framework for Mixed-Method Evaluation Designs
21 Sep 1989-Educational Evaluation and Policy Analysis (SAGE Publications)-Vol. 11, Iss: 3, pp 255-274
TL;DR: In recent years evaluators of educational and social programs have expanded their methodological repertoire with designs that include the use of both qualitative and quantitative methods as discussed by the authors, which can be classified into three categories: qualitative, quantitative, and qualitative.
Abstract: In recent years evaluators of educational and social programs have expanded their methodological repertoire with designs that include the use of both qualitative and quantitative methods. Such prac...
Citations
More filters
TL;DR: In this paper, the authors position mixed methods research (mixed research is a synonym) as the natural complement to traditional qualitative and quantitative research, and present pragmatism as offering an attractive philosophical partner for mixed method research.
Abstract: The purposes of this article are to position mixed methods research (mixed research is a synonym) as the natural complement to traditional qualitative and quantitative research, to present pragmatism as offering an attractive philosophical partner for mixed methods research, and to provide a framework for designing and conducting mixed methods research. In doing this, we briefly review the paradigm “wars” and incompatibility thesis, we show some commonalities between quantitative and qualitative research, we explain the tenets of pragmatism, we explain the fundamental principle of mixed research and how to apply it, we provide specific sets of designs for the two major types of mixed methods research (mixed-model designs and mixed-method designs), and, finally, we explain mixed methods research as following (recursively) an eight-step process. A key feature of mixed methods research is its methodological pluralism or eclecticism, which frequently results in superior research (compared to monomethod resear...
11,330 citations
Cites background or methods from "Toward a Conceptual Framework for M..."
...As noted by Greene et al. (1989), there are five major purposes or rationales for conducting 21OCTOBER 2004...
[...]
...Also, research in a content domain that is dominated by one method often can be better informed by the use of multiple methods (e.g., to give a read on methods-induced bias, for corroboration, for complimentarity, for expansion; see Greene et al., 1989)....
[...]
TL;DR: In this paper, the authors examined how the field of mixed methods currently is being defined and discussed the criteria of demarcation in mixed methods research, and concluded that mixed methods are one of the three major research paradigms.
Abstract: The purpose of this article is to examine how the field of mixed methods currently is being defined. The authors asked many of the current leaders in mixed methods research how they define mixed methods research. The authors provide the leaders' definitions and discuss the content found as they searched for the criteria of demarcation. The authors provide a current answer to the question, What is mixed methods research? They also briefly summarize the recent history of mixed methods and list several issues that need additional work as the field continues to advance. They argue that mixed methods research is one of the three major “research paradigms” (quantitative research, qualitative research, and mixed methods research). The authors hope this article will contribute to the ongoing dialogue about how mixed methods research is defined and conceptualized by its practitioners.
6,049 citations
Cites methods from "Toward a Conceptual Framework for M..."
...By examining published research, Greene, Caracelli, and Graham (1989) inductively identified the following five broad purposes or rationales of mixed methodological studies: (a) triangulation (i.e., seeking convergence and corroboration of results from different methods studying the same…...
[...]
01 Jan 2014
TL;DR: In this article, the authors used interpretative phenomenological analysis (IPA) to analyse 8 participants' experiences of rejection sensitivity and found that rejection sensitivity is the same concept as abandonment anxiety.
Abstract: Research demonstrates that rejection sensitivity develops through early, continuing, or acute experiences of rejection from caregivers and significant others. Rejection sensitivity refers to individuals who anxiously or angrily expect, readily perceive, and intensely react to rejection. The question regarding why rejection is feared by rejection sensitive individuals remains unanswered by existing rejection sensitivity literature. Therefore, the current study answers this question using interpretative phenomenological analysis (IPA) to analyse 8 participants' experiences of rejection sensitivity. Four superordinate themes emerged: `experiences of parenting'; `impact of rejection'; `coping with the concept of rejection'; and `identity'. The primary fundamental finding indicates that rejection sensitivity is the same concept as abandonment anxiety. Participants in the current study demonstrate both rejection sensitivity and abandonment anxiety. Furthermore, the origins and characteristics of both concepts are identified as the same. Therefore, these findings indicate that rejection is feared for the same reason that abandonment is feared. In childhood, abandonment is experienced as terrifying and therefore defences are adopted to avoid further abandonment. The concept of `past in present' means that childhood feelings can be timelessly re-experienced in adulthood as actual and unchanged. Therefore, later rejection situations are perceived as abandonment and accordingly alert an individual to impending danger. As a result, rejection is feared because it is perceived as abandonment and as a threat to survival. This finding is fundamental to the fields of rejection sensitivity and abandonment anxiety, in terms of research and therapeutic work with clients. Integrating existing literature provides much greater depth of knowledge and support for these concepts. Recommended therapeutic approaches for abandonment anxiety can also inform interventions for rejection sensitive clients. Findings also suggest that participants experience annihilation anxiety in relation to perceived rejection, which further increases fear. Clinical applications and implications with respect to the findings arc discussed.
3,365 citations
TL;DR: The authors conducted a content analysis of 232 social science articles in which the two were combined, and found that on the quantitative side structured interview and questionnaire research within a cross-sectional design tends to predominate, while on the qualitative side the semi-structured interview within a single-subject design tends not to dominate.
Abstract: This article seeks to move beyond typologies of the ways in which quantitative and qualitative research are integrated to an examination of the ways that they are combined in practice. The article is based on a content analysis of 232 social science articles in which the two were combined. An examination of the research methods and research designs employed suggests that on the quantitative side structured interview and questionnaire research within a cross-sectional design tends to predominate, while on the qualitative side the semi-structured interview within a cross-sectional design tends to predominate. An examination of the rationales that are given for employing a mixed-methods research approach and the ways it is used in practice indicates that the two do not always correspond. The implications of this finding for how we think about mixed-methods research are outlined.
3,255 citations
Cites background or methods or result from "Toward a Conceptual Framework for M..."
...Complementarity: ‘seeks elaboration, enhancement, illustration, clarification of the results from one method with the results from another’ (Greene et al., 1989: 259)....
[...]
...In their analysis of evaluation research articles, Greene et al. (1989) coded each article in terms of a primary and a secondary rationale, a procedure that was also employed by Niglas (2004)....
[...]
...Initiation: ‘seeks the discovery of paradox and contradiction, new perspectives of [sic] frameworks, the recasting of questions or results from one method with questions or results from the other method’ (Greene et al., 1989: 259)....
[...]
...Development: ‘seeks to use the results from one method to help develop or inform the other method, where development is broadly construed to include sampling and implementation, as well as measurement decisions’ (Greene et al., 1989: 259)....
[...]
...Expansion: ‘seeks to extend the breadth and range of enquiry by using different methods for different inquiry components’ (Greene et al., 1989: 259)....
[...]
TL;DR: The authors introduce a contingency framework that relates prior work to the design of a research project, paying particular attention to the question of when to mix qualitative and quantitative data in a single research paper.
Abstract: Methodological fit, an implicitly valued attribute of high-quality field research in organizations, has received little attention in the management literature. Fit refers to internal consistency among elements of a research project—research question, prior work, research design, and theoretical contribution. We introduce a contingency framework that relates prior work to the design of a research project, paying particular attention to the question of when to mix qualitative and quantitative data in a single research paper. We discuss implications of the framework for educating new field researchers.
2,650 citations
Cites background from "Toward a Conceptual Framework for M..."
...At the same time, integrating qualitative and quantitative data effectively can be difficult (e.g., Greene et al., 1989), and there is a risk of losing the strengths of either approach on its own....
[...]
References
More filters
TL;DR: This transmutability of the validation matrix argues for the comparisons within the heteromethod block as the most generally relevant validation data, and illustrates the potential interchangeability of trait and method components.
Abstract: Content Memory (Learning Ability) As Comprehension 82 Vocabulary Cs .30 ( ) .23 .31 ( ) .31 .31 .35 ( ) .29 .48 .35 .38 ( ) .30 .40 .47 .58 .48 ( ) As judged against these latter values, comprehension (.48) and vocabulary (.47), but not memory (.31), show some specific validity. This transmutability of the validation matrix argues for the comparisons within the heteromethod block as the most generally relevant validation data, and illustrates the potential interchangeability of trait and method components. Some of the correlations in Chi's (1937) prodigious study of halo effect in ratings are appropriate to a multitrait-multimethod matrix in which each rater might be regarded as representing a different method. While the published report does not make these available in detail because it employs averaged values, it is apparent from a comparison of his Tables IV and VIII that the ratings generally failed to meet the requirement that ratings of the same trait by different raters should correlate higher than ratings of different traits by the same rater. Validity is shown to the extent that of the correlations in the heteromethod block, those in the validity diagonal are higher than the average heteromethod-heterotrait values. A conspicuously unsuccessful multitrait-multimethod matrix is provided by Campbell (1953, 1956) for rating of the leadership behavior of officers by themselves and by their subordinates. Only one of 11 variables (Recognition Behavior) met the requirement of providing a validity diagonal value higher than any of the heterotrait-heteromethod values, that validity being .29. For none of the variables were the validities higher than heterotrait-monomethod values. A study of attitudes toward authority and nonauthority figures by Burwen and Campbell (1957) contains a complex multitrait-multimethod matrix, one symmetrical excerpt from which is shown in Table 6. Method variance was strong for most of the procedures in this study. Where validity was found, it was primarily at the level of validity diagonal values higher than heterotrait-heteromethod values. As illustrated in Table 6, attitude toward father showed this kind of validity, as did attitude toward peers to a lesser degree. Attitude toward boss showed no validity. There was no evidence of a generalized attitude toward authority which would include father and boss, although such values as the VALIDATION BY THE MULTITRAIT-MULTIMETHOD MATRIX
15,795 citations
"Toward a Conceptual Framework for M..." refers background or methods in this paper
...Variations within this triangulation purpose include Campbell and Fiske's (1959) advocacy of multiple methods to evaluate discriminant as well as convergent validity, and Mark and Shotland's (1987) idea of using multiple methods to bracket rather than converge on the correct answer. This idea of triangulation with a confidence interval is drawn from Reichardt and Gollob (1987). In a complementarity mixed-method study, qualitative and quantitative methods are used to measure overlapping but also different facets of a phenomenon, yielding an enriched, elaborated understanding of that phenomenon....
[...]
...Variations within this triangulation purpose include Campbell and Fiske's (1959) advocacy of multiple methods to evaluate discriminant as well as convergent validity, and Mark and Shotland's (1987) idea of using multiple methods to bracket rather than converge on the correct answer....
[...]
...The methods characteristic represents the degree to which the qualitative and quantitative methods selected for a given study are similar to or different from one another in form, assumptions, strengths, and limitations or biases (as argued by Campbell & Fiske, 1959)....
[...]
...(See Campbell & Fiske, 1959; Denzin, 1978; Webb, Campbell, Schwartz, & Sechrest, 1966; see also Mathison , 1988, for an excellent discussion of triangulation from these same sources.)...
[...]
TL;DR: In this paper, the authors focus on focusing and bounding the collection of data, focusing on within-site and cross-site analysis, and drawing and verifying conclusions of the results.
Abstract: Part One: Introduction Part Two: Focusing and Bounding the Collection of Data Part Three: Analysis During Data Collection Part Four: Within-Site Analysis Part Five: Cross-Site Analysis Part Six: Matrix Displays: Some General Suggestions Part Seven: Drawing and Verifying Conclusions Part Eight: Concluding Remarks
11,004 citations
Book•
01 Jan 1977
TL;DR: The Research Act as discussed by the authors is a textbook for methods courses and a major contribution to sociological theory, which teaches students the principles of research and how to construct and test theories by presenting four major approaches to experimentation: survey research, participant observation, life histories, and symbolic interaction.
Abstract: At once a unique textbook for methods courses and a major contribution to sociological theory, this book teaches students the principles of research and how to construct and test theories. It brings coherence to the study of methods by presenting four major approaches to experimentation: survey research, participant observation, life histories, and unobtrusive measures from a single theoretical point of view, symbolic interaction. It demonstrates the need for a synthesis between theory and methods, and shows how different methods limit and aff ect research results.Denzin's argues that no single method, theory, or observer can capture all that is relevant or important in reality. He argues for the use of triangulation and for a view of theory and methods as "concept sensitizers." His approach enables sociologists to acquire specifi c facts about a particular situation while simultaneously elevating these to the level of shared meaning.The author shows students how to proceed with research, bringing sharply into focus the possibilities and their limitations. Since his view is integrated rather than eclectic, this is much more than a "how to do it" manual. Denzin points out aspects of research that fall outside the scope of a given method yet aff ect results, and emphasizes the need to employ several methods to cross-check each other. The Research Act covers all the content of conventional methods courses. The presentation is exciting and imaginative, and provides a thorough review of major sociological methods, a cogent statement about approaches to sociological inquiry, and a source from which a understanding of the problems of research can be derived.
8,572 citations
TL;DR: There is a distinct tradition in the literature on social science research methods that advocates the use of multiple methods as mentioned in this paper, which is usually described as one of convergent methodology, multimethod/multitrait (Campbell and Fiske, 1959), convergent validation or, what has been called "triangulation".
Abstract: December 1979, volume 24 There is a distinct tradition in the literature on social science research methods that advocates the use of multiple methods. This form of research strategy is usually described as one of convergent methodology, multimethod/multitrait (Campbell and Fiske, 1959), convergent validation or, what has been called "triangulation" (Webb et al., 1 966). These various notions share the conception that qualitative and quantitative methods should be viewed as complementary rather than as rival camps. In fact, most textbooks underscore the desirability of mixing methods given the strengths and weaknesses found in single method designs.
7,449 citations
TL;DR: How quantitative and qualitative methods can be combined in a single evaluation study to better understand the phenomenon in question is discussed.
Abstract: This article discusses how quantitative and qualitative methods can be combined in a single evaluation study to better understand the phenomenon in question. Three perspectives on combining methods...
941 citations