Showing papers in "Organizational Research Methods in 2008"
TL;DR: The four articles in this special section onMeta-analysis illustrate some of the complexities entailed in meta-analysis methods and contributes both to advancing this methodology and to the increasing complexities that can befuddle researchers.
Abstract: During the past 30 years, meta-analysis has been an indispensable tool for revealing the hidden meaning of our research literatures. The four articles in this special section on meta-analysis illus...
20,272 citations
TL;DR: The use of interrater reliability (IRR) and intra-arrater agreement (IRA) indices has increased dramatically during the past 20 years as mentioned in this paper, at least in part because of the increased role of multilevel modeling techniques (e.g., hierarchical linear modeling and multi-level structural equation modeling) in organizational research.
Abstract: The use of interrater reliability (IRR) and interrater agreement (IRA) indices has increased dramatically during the past 20 years. This popularity is, at least in part, because of the increased role of multilevel modeling techniques (e.g., hierarchical linear modeling and multilevel structural equation modeling) in organizational research. IRR and IRA indices are often used to justify aggregating lower-level data used in composition models. The purpose of the current article is to expose researchers to the various issues surrounding the use of IRR and IRA indices often used in conjunction with multilevel models. To achieve this goal, the authors adopt a question-and-answer format and provide a tutorial in the appendices illustrating how these indices may be computed using the SPSS software.
2,775 citations
TL;DR: In this article, the authors extended MacKinnon and colleagues' work by conducting a simulation that examines the distribution of mediation and suppression effects of latent variables with structural equation modeling, and the properties of confidence intervals developed from eight different methods.
Abstract: Because of the importance of mediation studies, researchers have been continuously searching for the best statistical test for mediation effect. The approaches that have been most commonly employed include those that use zero-order and partial correlation, hierarchical regression models, and structural equation modeling (SEM). This study extends MacKinnon and colleagues (MacKinnon, Lockwood, Hoffmann, West, & Sheets, 2002; MacKinnon, Lockwood, & Williams, 2004, MacKinnon, Warsi, & Dwyer, 1995) works by conducting a simulation that examines the distribution of mediation and suppression effects of latent variables with SEM, and the properties of confidence intervals developed from eight different methods. Results show that SEM provides unbiased estimates of mediation and suppression effects, and that the bias-corrected bootstrap confidence intervals perform best in testing for mediation and suppression effects. Steps to implement the recommended procedures with Amos are presented.
1,574 citations
TL;DR: In this article, the authors compared three alternate effect size estimates for repeated measurements in both treatment and control groups, and found that the alternate measures of effect size were less accurate than the original measures.
Abstract: Previous research has recommended several measures of effect size for studies with repeated measurements in both treatment and control groups. Three alternate effect size estimates were compared in...
1,427 citations
TL;DR: In a three-path mediational model, two mediators intervene in a series between an independent and a dependent variable as mentioned in this paper, and methods of testing for mediation in such a model are generalized from the mor...
Abstract: In a three-path mediational model, two mediators intervene in a series between an independent and a dependent variable. Methods of testing for mediation in such a model are generalized from the mor...
956 citations
TL;DR: A study of qualitative researchers who have submitted and/or published their research in top-tier North American organizational and management journals reveals the evaluative criteria-in-use at the qualitative researchers as mentioned in this paper.
Abstract: A study of qualitative researchers who have submitted and/or published their research in top-tier North American organizational and management journals reveals the evaluative criteria-in-use at the...
508 citations
TL;DR: QCA's ability to examine the potential interdependence and complexity among effects through a study of how industry, corporate, and business- unit attributes combine in determining business-unit performance is demonstrated.
Abstract: The authors present qualitative comparative analysis (QCA) as a viable method for strategic management research. Specifically, they demonstrate its ability to examine the potential interdependence ...
334 citations
TL;DR: In this article, the authors demonstrate that the validity of assumed mediation models is highly suspect when they are based on the findings of nonexperimental research and propose strategies for conducting research using these and other assumptions.
Abstract: Tests of assumed mediation models are common in the organizational sciences. However, the validity of inferences about mediation is a function of experimental design and the setting of a study. Regrettably, most tests of mediation have relied on the application of so-called ``causal modeling'' techniques to data from nonexperimental studies. As we demonstrate, inferences about the validity of assumed mediation models are highly suspect when they are based on the findings of nonexperimental research. One of the many reasons for this is the failure of the model being tested to be consistent with reality. Valid research-based inferences about mediation are possible. However, inferences from such tests are most likely to be valid when they are based on research that uses randomized experimental designs. Strategies for conducting research using these and other designs are described. Finally, we offer a set of conclusions and recommendations that stem from our analysis.
284 citations
TL;DR: In this paper, the authors discuss the determination of quality in studies using grounded theory method (GTM) and concretely describe some misunderstandings associated with GTM and some malaises experienced with it.
Abstract: The authors discuss the determination of quality in studies using grounded theory method (GTM). They concretely describe some misunderstandings associated with GTM and some malaises experienced wit...
265 citations
TL;DR: This paper reviewed the author's history with mediation and explained why there is such interest in the topic and commented on the five articles in this feature topic, concluding that mediation is an important topic.
Abstract: This commentary reviews the author's history with mediation. In the second part, the author attempts to explain why there is such interest in the topic. In the third part, the author comments on the five articles in this feature topic.
253 citations
TL;DR: The authors apply the DICTION software to a sample of mission statements from 408 Association to Advance Collegiate Schools of Business colleges of business with a focus on gaining insights concerning how DICTIONS can be used in strategic management research utilizing content analysis.
Abstract: DICTION is a computer-aided content analysis program grounded in a number of theoretical bases in linguistic research. DICTION has a number of attractive features that could be used to analyze unique elements of language in narrative texts germane to strategic management research. The authors apply the DICTION software to a sample of mission statements from 408 Association to Advance Collegiate Schools of Business colleges of business with a focus on gaining insights concerning how DICTION can be used in strategic management research utilizing content analysis. They find significant differences in word usage in mission statements based on organizational characteristics, including business school performance. They conclude with suggestions for future research.
TL;DR: In this article, the authors present a threat to the validity of meta-analyses by duplicating study effects from the same data set and duplicated in more than one publication likely bias the aggregated effects in a meta-analysis.
Abstract: Study effects developed from the same data set and duplicated in more than one publication likely will bias the aggregated effects in a meta-analysis. This threat to the validity of meta-analyses i...
TL;DR: The authors review and critique the conduct and reporting of mediation analyses in 409 studies published in five leading organization studies journals over the past 25 years and find that the majority of inferences of full and partial mediation are based on testing procedures that deviate significantly from procedures recommended by statisticians.
Abstract: The authors review and critique the conduct and reporting of mediation analyses in 409 studies published in five leading organization studies journals over the past 25 years. The aim of the study is to learn from past practice and to use that knowledge to signal to researchers the importance of correctly applying mediation tests as well as to facilitate the valid testing of mediation models and the reporting of mediation results in future studies. The authors content code their sample for a wide range of characteristics and find that the majority of inferences of full and partial mediation are based on testing procedures that deviate significantly from procedures recommended by statisticians. In addition, the reporting of results is often incomplete and inefficient. The authors discuss and evaluate the findings of the study and make recommendations for future testing and reporting of results for mediation models.
TL;DR: This paper explored the use of criteria from the perspective of reviewers, editors, and/or authors in qualitative research and found that the making of quality is situated in methodological pluralism that occurs both in comparison with quantitative research and also within qualitative research.
Abstract: This Feature Topic contains four articles that address the determination of quality in qualitative research by exploring the use of criteria from the perspective of reviewers, editors, and/ or authors. In this introductory article, the authors assert that these explorations represent an important move away from employing listings of static criteria to adjudicate and develop qualitative research. In its place, we see the making of quality as situated in methodological pluralism that occurs both in comparison with quantitative research and also within qualitative research. This fact complicates and enriches the task of determining quality and also suggests ways forward for the academic community.
TL;DR: In this article, the authors outline the political and institutional structures that work to formulate operating norms that govern what is considered to be ''acceptable'' qualitative organizational research, and the quality indicators attached to foundational, quasi-foundational, and nonfoundational research orientations.
Abstract: Within this article, the authors outline the political and institutional structures that work to formulate operating norms that govern what is considered to be ``acceptable'' qualitative organizational research, and the quality indicators attached to foundational, quasi-foundational, and nonfoundational research orientations. They argue that encouraging a plurality of methods and representations will better position the field of organizational studies to address the most significant questions of our time. Located within this position, they call for a democratization of what counts in organizational research: a more considered and central space for nonhegemonic approaches to qualitative work. In so doing, they champion a moral-sacred epistemology that foregrounds ethical and moral concerns as underpinning both the purpose and the quality of the research.
TL;DR: Despite being a relatively young discipline, strategic management has grown dramatically in size and influence over the past few decades as discussed by the authors. As with any field, the findings generated generated within strategic...
Abstract: Despite being a relatively young discipline, strategic management has grown dramatically in size and influence over the past few decades. As with any field, the findings generated within strategic ...
TL;DR: In this article, the importance of examining the measurement equivalence of higher-order constructs from a theoretical perspective is discussed, and a small-scale simulation is presented to demonstrate that importance.
Abstract: Recent developments in the literature provide us with a better understanding of the importance of and procedures for testing measurement equivalence in organizational research. However, whereas many constructs in organizational research are multidimensional and multifaceted, procedures for testing the measurement equivalence of higher-order constructs are not commonly known. This study first discusses the importance of examining the measurement equivalence of higher-order constructs from a theoretical perspective, and then demonstrates that importance with a small-scale simulation. Finally, the procedures for testing various forms of measurement equivalence with structural equation modeling for both firstand second-order constructs are clarified and illustrated with a numerical example.
TL;DR: In this article, the authors explore the potential for this methodology to be more widely used in strategic management research, using the example of a recently completed case study of strategic change in a large banking and financial services institution.
Abstract: Critical discourse analysis has become an increasingly popular methodology in organization and management studies. In this article, the authors explore the potential for this methodology to be more widely used in strategic management research. They begin by identifying three research approaches that, to a greater or lesser extent, share a concern with the relationship between language and the formulation and implementation of strategy—strategy as a system of shared meaning, strategy as text and talk, and strategy as truth. They then discuss how critical discourse analysis can be used to extend and develop these approaches by exploiting their underlying complementarities. Finally, using the example of a recently completed case study of strategic change in a large banking and financial services institution, they explore the practical implications of applying critical discourse analysis in strategic management research.
TL;DR: It is argued that some limitations of post hoc qualitative methods in deductive theory testing can be alleviated by using a prospective case study design, where researchers formulate a set of theory-based hypotheses and then test these hypotheses at a predetermined follow-up time in the future by comparing them with the observed outcomes using pattern matching or a similar technique.
Abstract: The article advances a prospective case study design in social sciences as an alternative to traditional post hoc case study research in deductive theory testing. It is argued that some limitations of post hoc qualitative methods in deductive theory testing can be alleviated by using a prospective case study design, where researchers formulate a set of theory-based hypotheses in respect to the evolution of an ongoing social process and then test these hypotheses at a predetermined follow-up time in the future by comparing them with the observed outcomes using pattern matching or a similar technique. Some challenges of conducting prospective research in social sciences are discussed. An illustration of the application of this method in deductive testing of two competing theories is provided.
TL;DR: In this paper, the authors revisit the history of mediational inference in social and organizational sciences and illustrate how mediational inferences have been evident for more than a century Some of the important milestones and the evolution of thought leading to today are chronicled.
Abstract: This introductory article for the feature topic on mediation is designed to provide a context for the articles that follow Specifically, the authors revisit the history of mediational inferences in social and organizational sciences and illustrate how mediational inferences have been evident for more than a century Some of the important milestones and the evolution of thought leading to today are chronicled Then a modern-day framework for understanding mediational inferences is outlined that serves to highlight how the articles included in this feature topic contribute to the continued evolution of the bases of mediational inferences The article concludes with the authors' vision of what may be fruitful directions for future research
TL;DR: In this article, the authors draw on their own experiences and on the literature on auto-ethnography to illustrate the strengths and challenges of researching back home, arguing that autoethnographic approaches have four important strengths: ease of access, reduced resource requirements, ease of establishing trust and rapport, and reduced problems with translation.
Abstract: The challenges facing international management researchers conducting research in foreign contexts are increasingly well understood. However, for a growing group of researchers, the problem is very different: Rather than being foreign researchers researching in an unfamiliar context, they are insiders conducting research in their own cultural context for publication in international journals. In this article, the authors draw on their own experiences and on the literature on autoethnography to illustrate the strengths and challenges of researching “back home.” In particular, they argue that autoethnographic approaches have four important strengths— ease of access, reduced resource requirements, ease of establishing trust and rapport, and reduced problems with translation—but at the same time pose three important challenges— lack of critical distance, role conflict, and the limits of serendipity.
TL;DR: In this article, Monte Carlo simulations were conducted to compare the Hedges and Olkin, the Hunter and Schmidt, and a refinement of the Aguinis and Pierce meta-analytic approaches for estimating moderating effects of categorical variables.
Abstract: The authors conducted Monte Carlo simulations to compare the Hedges and Olkin, the Hunter and Schmidt, and a refinement of the Aguinis and Pierce meta-analytic approaches for estimating moderating effects of categorical variables. The simulation examined binary moderator variables (e.g., gender—male, female; ethnicity—majority, minority). The authors compared the three meta-analytic methods in terms of their point estimation accuracy and Type I and Type II error rates. Results provide guidelines to help researchers choose among the three meta-analytic techniques based on theory (i.e., exploratory vs. confirmatory research) and research design considerations (i.e., degree of range restriction and measurement error).
TL;DR: The results of a two-part study to illustrate the effect of choice of meta-analytic model on study conclusions suggest that random-effects procedures represent the best initial choice when conducting a meta-analysis.
Abstract: Fixed- and random-effects models represent two different approaches to analyzing and understanding data with meta-analysis. The current article describes the results of a two-part study to illustrate the effect of choice of meta-analytic model on study conclusions. Part 1 illustrates the effect of model choice by analyzing data simulated to conform to either fixed- or random-effects scenarios with both fixed- and random-effects methods of data analysis. Part 2 uses two published meta-analyses to show that methodological choices, in this case mainly the choice of fixed- or random-effects models, affect estimates both of mean effect size and of the random-effects variance component (REVC). Overall, results suggest that random-effects procedures represent the best initial choice when conducting a meta-analysis.
TL;DR: In this article, the authors used the work of Smircich and Chesser on direct and meta-perspectives to test the correspondence between self-reported and supervisory ratings of performance.
Abstract: This article uses the work of Smircich and Chesser on direct and meta-perspectives (i.e., points of view) to test the correspondence between self-reported and supervisory ratings of performance. Results from two field studies indicate a higher level of self-supervisor rating correlation when using a common perspective. Similar results were obtained using an alternative measure of rating correspondence. The results provide evidence that researchers using self-reports of performance as surrogates for archival performance data should ask respondents for their supervisor's assessment of their performance rather than for their own assessment. The practical implications of these results for field research are discussed.
TL;DR: In this article, content analysis of textual archival data facilitates elaboration of adolescent theories, including scientific robustness, relationship identification, and boundary establishment, using an empirical example from a health care study.
Abstract: This article shows how content analysis of textual archival data facilitates elaboration of adolescent theories. Drawing on the framework of a well-developed theory offered by Bacharach, the authors explain how specific protocols in content analysis can facilitate elaboration of three theoretical elements underdeveloped in adolescent theories: scientific robustness, relationship identification, and boundary establishment. Using an empirical example from a health care study, they illustrate the use of content analysis protocols to elaborate some theoretical elements of the attention-based view of the firm. The article contributes to content analysis and more generally to research methodology by specifying closer linkages between theory elaboration and content analysis.
TL;DR: In this article, an individual's demographic similarity to a group is defined as a type of cross-level theory labeled as an individual-within-the-group model, which is a cross-layer theory.
Abstract: Relational demography within groups, or an individual's demographic similarity to a group, is a type of cross-level theory labeled as an individual-within-the-group model. Previous researchers used...
TL;DR: This paper found that relatively dissatisfied employees were more likely to provide comments than their more satisfied counterparts and that open-ended responses were disproportionately negative in tone and tended to echo commenters' closed-ended satisfaction ratings.
Abstract: Recent technologies have reduced some of the major barriers to capturing, coding, and analyzing qualitative data from survey respondents. This has prompted a renewed interest in including openended questions on employee surveys and a corresponding need to better understand the potential biases of personnel who choose to provide comments. The present study used data from a climate survey (N = 661) to empirically examine qualitative comments and their relationship with quantitative survey ratings. Results revealed that relatively dissatisfied employees were more likely to provide comments than their more satisfied counterparts. Moreover, open-ended responses were disproportionately negative in tone and tended to echo commenters’ closed-ended satisfaction ratings. For most survey dimensions studied, the length of comments increased as they became more negative in tone. Finally, the data revealed very few demographic differences between respondents who provided comments and those who did not.
TL;DR: In this paper, the authors examined the adjustment function in the context of a 2 x 2 design with two manipulated variables and showed that this leads to improper control of the mediator and to biased estimates of the model parameters.
Abstract: In a simple mediation model, the effect of a manipulated variable X on a dependent variable Y over and above the effect of the mediator Me can be estimated by regressing Y on X and Me. The impact of X on Y in such a model is adjusted for the relationship both between X and Me and between Me and Y. The authors examine the adjustment function in the context of a 2 x 2 design with two manipulated variables. In such a situation, the mediator could be affected by either one of the main effects and/or the interaction between two manipulated variables. To adjust for the impact of the mediator, a standard procedure has been to rely on an ANCOVA that includes only the mediator. The authors show, both analytically and with simulations, that this leads to improper control of the mediator and to biased estimates of the model parameters.
TL;DR: In this article, the problem of estimating the squared multiple correlation coefficient and the squared cross-validity coefficient under the assumption that the response and predictor variables have a joint multinormal distribution was considered.
Abstract: The sample squared multiple correlation coefficient is widely used for describing the usefulness of a multiple linear regression model in many areas of science. In this article, the author considers the problem of estimating the squared multiple correlation coefficient and the squared cross-validity coefficient under the assumption that the response and predictor variables have a joint multinormal distribution. Detailed numerical investigations are conducted to assess the exact bias and mean square error of the proposed modifications of established estimators. Notably, the positive-part Pratt estimator and the synthesis of Browne and positive-part Pratt estimators are recommended in the estimation of squared multiple correlation coefficient and squared cross-validity coefficient, respectively, for their overall advantages of incurring the least amount of statistical discrepancy and computational requirement.
TL;DR: The application of the repertory grid technique in the strategy field has been noteworthy as discussed by the authors, however, all have been imprisoned by their adherence to strict element choice based on grid protocol.
Abstract: Previous application of the repertory grid technique in the strategy field has been noteworthy. However, all have been imprisoned by their adherence to strict element choice based on grid protocol....