Showing papers in "Organizational Research Methods in 2013"
TL;DR: In this paper, a systematic approach to new concept development and grounded theory articulation that is designed to bring "qualitative rigor" to the conduct and presentation of inductive research is presented.
Abstract: For all its richness and potential for discovery, qualitative research has been critiqued as too often lacking in scholarly rigor. The authors summarize a systematic approach to new concept development and grounded theory articulation that is designed to bring “qualitative rigor” to the conduct and presentation of inductive research.
6,189 citations
TL;DR: Although the emphasis is on regression, structural equation modeling, and multilevel modeling, the general framework forms the basis for a research agenda regarding outliers in the context of other data-analytic approaches.
Abstract: The presence of outliers, which are data points that deviate markedly from others, is one of the most enduring and pervasive methodological challenges in organizational science research. We provide evidence that different ways of defining, identifying, and handling outliers alter substantive research conclusions. Then, we report results of a literature review of 46 methodological sources (i.e., journal articles, book chapters, and books) addressing the topic of outliers, as well as 232 organizational science journal articles mentioning issues about outliers. Our literature review uncovered (a) 14 unique and mutually exclusive outlier definitions, 39 outlier identification techniques, and 20 different ways of handling outliers; (b) inconsistencies in how outliers are defined, identified, and handled in various methodological sources; and (c) confusion and lack of transparency in how outliers are addressed by substantive researchers. We offer guidelines, including decision-making trees, that researchers can...
730 citations
TL;DR: It is argued that direct quantitative approaches, largely represented by computational modeling or agent-based simulation, have much to offer with respect to illuminating the mechanisms of emergence as a dynamic process, and illustrated how indirect and direct approaches can be complementary and, appropriately integrated, have the potential to substantially advance theory and research.
Abstract: Multilevel theory and research have advanced organizational science but are limited because the research focus is incomplete. Most quantitative research examines top-down, contextual, cross-level relationships. Emergent phenomena that manifest from the bottom up from the psychological characteristics, processes, and interactions among individuals—although examined qualitatively—have been largely neglected in quantitative research. Emergence is theoretically assumed, examined indirectly, and treated as an inference regarding the construct validity of higher level measures. As a result, quantitative researchers are investigating only one fundamental process of multilevel theory and organizational systems. This article advances more direct, dynamic, and temporally sensitive quantitative research methods designed to unpack emergence as a process. We argue that direct quantitative approaches, largely represented by computational modeling or agent-based simulation, have much to offer with respect to illuminatin...
335 citations
TL;DR: It is concluded that the method is widely misunderstood, and the results cast strong doubts on its effectiveness for building and testing theory in organizational research.
Abstract: Partial least squares path modeling (PLS) was developed in the 1960s and 1970s as a method for predictive modeling. In the succeeding years, applied disciplines, including organizational and manage...
320 citations
TL;DR: This article offers freely available software that computes and compares all of these indices with one another, computes associated bootstrapped confidence intervals, and does so for any number of predictors so long as the correlation matrix is positive definite.
Abstract: Multiple linear regression (MLR) remains a mainstay analysis in organizational research, yet intercorrelations between predictors (multicollinearity) undermine the interpretation of MLR weights in terms of predictor contributions to the criterion. Alternative indices include validity coefficients, structure coefficients, product measures, relative weights, all-possible-subsets regression, dominance weights, and commonality coefficients. This article reviews these indices, and uniquely, it offers freely available software that (a) computes and compares all of these indices with one another, (b) computes associated bootstrapped confidence intervals, and (c) does so for any number of predictors so long as the correlation matrix is positive definite. Other available software is limited in all of these respects. We invite researchers to use this software to increase their insights when applying MLR to a data set. Avenues for future research and application are discussed.
257 citations
TL;DR: The propensity score method (PSM)—which has previously been widely employed in social science disciplines such as public health and economics—is introduced to the management field and a procedure for applying the PSM to estimate the causal effects of intervention is developed.
Abstract: Evidence-based management requires management scholars to draw causal inferences. Researchers generally rely on observational data sets and regression models where the independent variables have not been exogenously manipulated to estimate causal effects; however, using such models on observational data sets can produce a biased effect size of treatment intervention. This article introduces the propensity score method (PSM)—which has previously been widely employed in social science disciplines such as public health and economics—to the management field. This research reviews the PSM literature, develops a procedure for applying the PSM to estimate the causal effects of intervention, elaborates on the procedure using an empirical example, and discusses the potential application of the PSM in different management fields. The implementation of the PSM in the management field will increase researchers’ ability to draw causal inferences using observational data sets.
213 citations
TL;DR: The Edwards and Lambert (2007) approach to integrating moderation and mediation to Structural Equation Modeling (SEM) is extended by constructing a one-stop procedure, and the new LMS approach to latent variable interactions implemented in Mplus is discussed.
Abstract: It is increasingly common to test hypotheses combining moderation and mediation. Structural equation modeling (SEM) has been the favored approach to testing mediation hypotheses. However, the biggest challenge to testing moderation hypotheses in SEM was the complexity underlying the modeling of latent variable interactions. We discuss the latent moderated structural equation procedure (LMS) approach to specifying latent variable interactions, which is implemented in Mplus, and offer a simple and accessible way of testing combined moderation and mediation hypotheses using SEM. To do so, we provide sample code for six commonly encountered moderation and mediation cases and relevant equations that can be easily adapted to researchers’ data. By articulating the similarities in the two different approaches, discussing the combination of moderation and mediation, we also contribute to the research methods literature.
152 citations
TL;DR: The authors apply a framework to develop organizational-level operationalizations of individual-level constructs using the psychological capital construct as an example, and demonstrate how computer-aided text analysis might be utilized to facilitate construct elevation while ensuring proper validation.
Abstract: Applying individual-level constructs to higher levels of analysis can be a fruitful practice in organizational research. Although this practice is beneficial in developing and testing theory, there are measurement and validation concerns that, if improperly addressed, may threaten the validity and utility of the research. This article illustrates how computer-aided text analysis might be utilized to facilitate construct elevation while ensuring proper validation. Specifically, we apply a framework to develop organizational-level operationalizations of individual-level constructs using the psychological capital construct as an example.
147 citations
TL;DR: In this paper, a new cluster-based approach, average silhouette width (ASW), was proposed to identify the number of subgroups and subgroup membership in a team, and compared with 1,400 simulated teams with varying properties.
Abstract: Team faultlines—hypothetical dividing lines based on member attributes that split a team into relatively homogeneous subgroups—influence team processes across contexts, as recent meta-analytic findings show. We review the available faultline measures with regard to their properties and identify several limitations, including dealing with more than two subgroups. We thus propose a new cluster-based approach, average silhouette width (ASW), that identifies the number of subgroups and subgroup membership. We then compare the measures with 1,400 simulated teams with varying properties and investigate their factor structure and their behavior under missing values. We also investigate the predictive validity of the measures with data from real work teams. Results show that different measures respond to different team features in different ways but that most of them load on two correlated factors. Taken together, the ASW measure had the most favorable attributes and was the only measure that accurately determine...
121 citations
TL;DR: In this paper, the authors proposed a framework for organizational performance that includes three dimensions: accounting dimension, accounting dimension and organizational performance dimension, which is a fundamental construct in strategic management.
Abstract: Organizational performance is a fundamental construct in strategic management. Recently, researchers proposed a framework for organizational performance that includes three dimensions: accounting r...
117 citations
TL;DR: Agent-based simulation models can reproduce the interactions between members of an organization or between different organizations in an artificial environment where "agents" make decisions and communicate with one another.
Abstract: Agent-based simulation models can reproduce the interactions between members of an organization or between different organizations in an artificial environment where “agents” make decisions and communicate with one another. This article discusses possible applications to core issues in organization science and provides an introductory guide to simulation platforms. Agent-based modeling requires writing computer code, a skill that, if properly mastered, may turn into a career opportunity.
TL;DR: This work extends Evans’s (1985) pioneering work, and the more recent works by Ostroff, Kinicki, and Clark (2002) and Siemsen, Roth, and Oliveira (2010), to delineate the influence of CMV in a two-level hierarchical linear model based on self-report data.
Abstract: Despite that common method variance (CMV) is widely regarded as a serious threat to the validity of findings based on self-reports, there is insufficient research on its confounding influence. We e...
TL;DR: In the two decades since storytelling was called the “sense-making currency of organizations,” storytelling scholarship has employed a wide variety of research methods, including storytelling diamond modes.
Abstract: In the two decades since storytelling was called the “sensemaking currency of organizations,” storytelling scholarship has employed a wide variety of research methods. The storytelling diamond mode...
TL;DR: In this article, the authors discuss the use of three prominent archival proxies (research and development intensity, patent counts, and patent citations) within recent articles in three leading journals.
Abstract: Archival proxies have long played a central role within strategic management research, but the degree to which archival proxies are construct valid measures of theoretical constructs remains a source of concern. In some cases, there does not appear to be a close association between an archival proxy and the construct that the proxy is meant to capture. In this brief commentary, we discuss the use of three prominent archival proxies (research and development intensity, patent counts, and patent citations) within recent articles in three leading journals. Each of these measures has been used to represent a wide variety of constructs, which creates challenges when interpreting findings. We then offer three suggestions for improving the use of archival proxies. Implementation of these suggestions would enhance knowledge development within the strategic management field.
TL;DR: The authors examined both random (survey errors) and systematic (social desirabi survey respondents) to examine both sleep and self-regulation in a survey of employees and survey respondents.
Abstract: Insomnia is a prevalent experience among employees and survey respondents. Drawing from research on sleep and self-regulation, we examine both random (survey errors) and systematic (social desirabi...
TL;DR: In this article, the authors argue that the difficulties in studying sensitive topics can be overcome through the underutilized method of field experiments, detail strategies for conducting high-quality experimental field studies, and offer suggestions for overcoming potential challenges in data collection and publishing.
Abstract: Organizational scholars study a number of sensitive topics that make employees and organizations vulnerable to unfavorable views. However, the typical ways in which researchers study these topics—via laboratory experiments and field surveys—can be laden with problems. In this article, the authors argue that the difficulties in studying sensitive topics can be overcome through the underutilized method of field experiments, detail strategies for conducting high-quality experimental field studies, and offer suggestions for overcoming potential challenges in data collection and publishing. As such, this article is designed to serve as a guide and stimulus for using the valuable methodological tool of field experiments.
TL;DR: The context of strategic management and measurement is described, the Context Topic articles are described, how the Feature Topic articles address critical issues are examined, and advice is given for authors and reviewers on this topic.
Abstract: Construct measurement is a cornerstone for research in any area. In the absence of information on the caliber of measurement practices within and across studies, it is extremely difficult to synthesize research and develop normative guidelines for managers. This Feature Topic aims to advance measurement practices by strategic management scholars. The Feature Topic includes six articles from a diverse array of perspectives. We describe the context of strategic management and mea- surement, examine how the Feature Topic articles address critical issues, and conclude with advice for authors and reviewers on this topic.
TL;DR: In this paper, three different yet related strategies for decomposing variance in higher-order multiple regression models are described: hierarchical F tests (a between-sets test), constrained relative importance analysis (a within-set test), and residualized relative importance analyses (between-and within-sets tests).
Abstract: The current article notes that the standard application of relative importance analyses is not appropriate when examining the relative importance of interactive or other higher order effects (e.g., quadratic, cubic). Although there is a growing demand for strategies that could be used to decompose the predicted variance in regression models containing such effects, there has been no formal, systematic discussion of whether it is appropriate to use relative importance statistics in such decompositions, and if it is appropriate, how to go about doing so. The purpose of this article is to address this gap in the literature by describing three different yet related strategies for decomposing variance in higher-order multiple regression models—hierarchical F tests (a between-sets test), constrained relative importance analysis (a within-sets test), and residualized relative importance analysis (a between- and within-sets test). Using a previously published data set, we illustrate the different types of inferen...
TL;DR: The authors examined how strategy scholars have measured and tested industry effects and found that there has been a decrease in the proportion of articles that do not control for industry effects at all and at the same time noting a significant increase in the number of single industry studies.
Abstract: The authors examine how strategy scholars have measured and tested industry effects. They report findings from three studies. First, they replicate the Dess, Ireland, and Hitt (1990) article on industry controls in strategic management research using a new sample of studies published during 2000 to 2009, finding that there has been a decrease in the proportion of articles that do not control for industry effects at all and at the same time noting a significant increase in the number of single-industry studies. Second, they employ a fine-grained content analysis of articles published in the Strategic Management Journal at three different points during the study period to identify the different ways that industry effects have been considered. Findings depict a myriad of highly diverse industry-level measures that researchers have applied. Third, they test the empirical implications of applying different measures of one particular industry characteristic, industry performance. They demonstrate that empirical...
TL;DR: In this paper, the authors present guidelines for applying interrater agreement indices to the vast majority of methodological and theoretical problems that organizational and applied psychology researchers encounter, focusing on important ways to extend the usage of inter-arrater agreement index beyond problems relating to the aggregation of individual level data.
Abstract: Currently, guidelines do not exist for applying interrater agreement indices to the vast majority of methodological and theoretical problems that organizational and applied psychology researchers encounter. For a variety of methodological problems, we present critical values for interpreting the practical significance of observed average deviation (AD) values relative to either single items or scales. For a variety of theoretical problems, we present null ranges for AD values, relative to either single items or scales, to be used for determining whether an observed distribution of responses within a group is consistent with a theoretically specified distribution of responses. Our discussion focuses on important ways to extend the usage of interrater agreement indices beyond problems relating to the aggregation of individual level data.
TL;DR: In this article, the authors adopt a construct validity lens to provide a critical reexamination of established corporate governance research, focusing on the body of work relying on the theoretical bases of...
Abstract: We adopt a construct validity lens to provide a critical reexamination of established corporate governance research. In particular, we focus on the body of work relying on the theoretical bases of ...
TL;DR: The concept of CEO duality describes the governance structure when a firm's chief executive officer also holds the position of chairperson of the board as discussed by the authors, and is central to theoretical perspectives on corporate go...
Abstract: CEO duality describes the governance structure when a firm’s chief executive officer also holds the position of chairman of the board. Duality is central to theoretical perspectives on corporate go...
TL;DR: Monte Carlo simulation results from several common random coefficient models are presented to highlight the scope and severity of the problem, and an analytic strategy is proposed to aid researchers in determining the underlying structure of the error covariance matrix.
Abstract: Organizational scientists increasingly focus on the dynamics of human behavior through longitudinal and event sampling methodologies. Random coefficient modeling such as hierarchical linear modelin...
TL;DR: In this article, the authors present an overview of bipartite networks, including refinements related to the projections of biparite networks onto unipartite network, with a focus on clustering coefficients, and approaches unique to bipartitite networks such as nestedness.
Abstract: Network-based research in the management field largely assumes one-mode (unipartite) networks, despite the widespread presence of two-mode (bipartite) networks. In empirical work, scholars usually project a bipartite network onto a unipartite network, ignoring issues related to the interdependence of ties and potential loss of information. Yet new advances in measures and methods related to bipartite networks in the fields of sociology, physics, and biology may make such tactics unnecessary. This article presents an overview of three research streams related to bipartite networks, namely, (a) refinements related to the projections of bipartite networks onto unipartite networks; (b) the extension of networks measures from unipartite networks to bipartite networks, with a focus on clustering coefficients; and (c) approaches unique to bipartite networks, such as nestedness. We apply these approaches and compare the findings of a traditional unipartite network analysis using both a simple example and a sample...
TL;DR: The authors analyzes reporting biases in regression analyses, focusing on how the Texas sharpshooter (TS) approach creates a predictor reporting bias (PRB) in primary studies and research syntheses.
Abstract: The author analyzes reporting biases in regression analyses. The consequences of researchers’ strategy to select significant predictors and omit nonsignificant predictors from regression analyses are examined, focusing on how this strategy—labeled the Texas sharpshooter (TS) approach—creates a predictor reporting bias (PRB) in primary studies and research syntheses. PRB was demonstrated in simulation studies when correlation coefficients from several primary regression studies with an underlying TS approach were aggregated in meta-analyses. Several important findings are noted. First, meta-analytical effect sizes of true effects can be overestimated because smaller, nonsignificant findings are omitted from regression models. Second, suppression effects of correlated predictor variables create biased effect size estimations for variables that are not related to the outcome. Finally, existing small effects are concealed, and between-study heterogeneity can be overestimated. Results show that PRB is contingent on sample size. While PRB is substantial in studies with small sample sizes (N 500) are analyzed. Preconditions and remedies for reporting biases in regression analyses are discussed.